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Mr. Emerson J. Elliott, Administrator 

National Center for Education Statistics 

Brown Building 

1200 19th Street, N.W. 

Washington, D.C. 20208 

Dear Mr. Elliott: 

The American Association for Counseling and Develop- 
ment (formerly the American Personnel and Guidance Association) 
is pleased to participate in the redesign of the National Center 
for Education Statistics elementary and secondary education 
data program. There are some important data collection/research 
issues that should be addressed in the future to provide 
invaluable information about learning and the rules and responsi- 
bilities of school personnel. 

Rather than provide lengthy papers on these issues, we 
have chosen to raise them in this letter, with some suggestions 
when possible. We would be pleased to discuss them in more de- 
tail with you and to devote staff and reader time to your 
redesign effort. 

Demographic Data 

In the past, there have been a number of problems in 
collecting data on noninstructional personnel. For example, 
elementary school counselors often serve more than one school, 
making it possible, perhaps even likely, that the same person 
is counted more than once. This problem is true of nurses, 
social workers and others as well. It is essential that the 
number of noninstructional personnel be reported on a district 
basis in terms of full-time equivalent units. It would also be 
valuable to know what the ratio of students to these different 
personnel is in each district. Because the quality of services 
provided to students rests upon the caliber of instruction as 
well as the types and extent of student needs that are met, it 
is essential to assess accurately the number and types of 
noninstructional personnel who provide services to students. 
AACD worked collaboratively with NCES in the preparation of 
Counselors in Local Education Agencies, Fall 1979 and Trends 
Since 1 970 (NCES 82-122b). We welcome the opportunity to 
participate in the updating of this research report. 
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Time and Task Analysis 

Recent studies on how teachers and students spend 
their classroom time have not only been instructive in under- 
standing what is happening in the classroom but have 1 3d educators 
to reassess how they spend their time. Similar studies for non- 
instructional personnel would be invaluable in assessing the 
types of services provided and the actual needs of students and 
faculty. Educators at all levels could use this information to: 

1. Gain a better understanding of the roles of nonin- 
structional personnel, including (but not limited to) school 
counselors, psychologists, social workers, nurses, speech and 
hearing specialists, librarians and medi3 specialists, adminis- 
trators and supervisors; 

2. Assess student needs and the degree to which they 
are being met by noninstructional personnel; 

3. Review and, if necessary, reformulate the roles 
and responsibilities of noninstructional personnel; and 

4. Develop a comprehensive approach b r <ed on reali- 
ties, not perceptions, of the appropriate tasks of all school 
personnel . 

Note: AACD is extremely interested in and willing to assist in 
any time and task studies directed to the study of the 
school counselor's role in elementary and secondary 
education. 

Demographic Data on Educational Personnel 

Information on education personnel, their ages, years 
of experiences, training and average length of stay is essential 
if we are to predict future personnel needs in a more systematic 
manner. These demographic data need to be correlated with 
expected student enrollment, based on birth and enrollment sta- 
tistics, to project future personnel needs. These projections 
should address the needs at least 5-10 years in advance to 
encourage/discourage youth and adults from pursuing training as 
educators. The projected teacher shortage illustrates the im- 
portance of such planning. Systematic, accurate information on 
current personnel and future needs is vital for all education 
personnel if we are to lessen or prevent personnel shortages or 
surpluses in the future. 
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Note: AACD can assist NCES in assessing the status of student 
enrollment in elementary and secondary counselor educa- 
tion programs. The association is also committed to 
gaining better supply and demand information for counsel- 
ing and pupil services personnel. 

Career Development 

Because the goal of education is not only to produce 
an educated citizenry but also to prepare future generations to 
enter the work force, it is essential that we assess students' 
career development needs, current activities of schools in foster- 
ing career development and what might be done to improve this 
aspect of school preparation. Such an assessment would require 
an indepth study, but that fact should not be a deterrent to it. 
The future of our children, our economy and our country is 
intricately tied to successful employment of future generations. 

Need for NCES Advisory Council 

We strongly recommend that NCES institute an advisory 
council composed of educators from diverse instructional and 
noninstructional backgrounds, as well as experts *n assessment, 
parents and students to provide advice on future survey efforts. 
This advisory council should be a formalized body, meeting to 
offer advice and direction to NCES. Diverse instructional and 
noninstructional personnel should be involved, regardless of 
the survey's focus, because few, if any, research areas are 
restricted to only one type of personnel. This advisory council 
should provide direction and advice and, in some cases, be in- 
volved in responding to the survey design, draft items and the 
method(s) for survey dissemination. Such involvement can help 
identify problems of overlap, unclear questions, misinterpreta- 
tion of terminology, gaps and other issues that ultimately 
reduce the validity and reliability of the data collected. 

Note: AACD is willing to identify staff, leader and profes- 
sional representatives to participate in such an NCES 
advisory council . 

Assessing School Counseling 

To support the belief that the Department and AACD 
should work in conjunction with each other, we have enclosed a 
list of research questions for assessing school counseling that 
we submitted in the fall of 1984 to the National Institute of 
Education as they formulated their priorities for the national 
center and laboratory competition. While these questions take 
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a more indepth look at research issues related to school counsel- 
ing, we think you might find them v>f value as you address specific 
questions related to scnool counseling. 

AACD stands ready to assist NCES as the center studies 
and develops ideas for collecting elementary and secondary educa- 
tion data. We can identify professionals who have the expertise 
you require as you look broadly at the education services we are 
currently providing and those we should be providing in the 
future. We welcome the opportunity to provide additional infor- 
mation or to answer any questions about our suggestions. We 
wish to be involved in future efforts. 

Sincerely yours, 



Patrick J. McQbndugh, Ed.O. > NCC / 




Executive Dir^cjtor 

FEB:LH 
Enclosure 
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8. How does the learning environment affect learning and can counselors 
change the learning environment to be more responsive to students 1 
needs? 

9. What is the role of the counselor in identifying children who are 
at-risk for personal, educational, or career problems? How can 
teachers and counselors work more effectively with children who 
have problems? 

10. What are the most effective approaches for the counselor in assisting 
students and their parents with career exploration and selection? 

11. What interventions are most effective in working with the following: 

• learning disabled 

• emotionally disturbed/deprived 

• physically handicapped 

• children of divorce 

t children in step-families 

t economically disadvantaged 

• substance abuse 

• child abuse 

• premature or delayed development 

• chronic truancy 

• underachievemont 

• social immaturity 

For further information, please contact Dr. Frank Burtnett or Dr. Sharon 
Alexander at AACD. 
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AACTE gathers data about higher education-based teacher education; consequently 
AACTE does not consider itself expert in the area of data collection concerning 
elementary or secondary education. Our membership has been involved, however, with a 
variety of data collection vehicles sponsored by the National Center for Education 
Statistics. On the basis of this involvement and the Association's data-collection 
efforts in another' sector, the following comments and observations are offered. 

General Areas of Data Needs 

In examining the documents distributed by NCES in this call for comment, and taking 
into consideration some long-term data needs of the teacher education community, we have 
identified four areas which should have a high priority for NCES: 

0 Teacher supply and demand . With increasing competition for scarce resources at 
both the preservice teacher education and inservice levels, it becomes imperative 
to have accurate current information on and future projections of teacher supply 
and demand. Such scarce resources need to be distributed so that teacher education 
programs can be responsive to the school personnel needs of local districts. 
Information on the teacher reserve pool (its size, mobility, and interest in 
returning to teaching), the retirement picture for the current workforce, accurate 
attrition figures, and "lateral entry" forecasts are needed both short-term and 
long-term. 

° Beginning teacher induction programs and inservice education . Data about programs 
in these areas are critical because they have implications for future program 
development. Estimates as to growth (or lack thereof) of inservice and beginning 
teacher induction programs for teachers will allow teacher educators to prepare 
intelligently to assist school districts wi th the continuing education of teachers. 
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° Data about teacher testing programs . To a large degree, the credibility of teacher 
preparation institutions is on the line with teacher tests. Despite the perceived 
inadequacies of the cu^ent tests, the quality of schools, colleges and departments 
of education is likely to be judged based on a variety of competency tests. If 
support for rigorous and demanding tests can be demonstrated, a "professional 
school mentality" may start to develop. In other professions, e.g., accounting and 
law, it is normal for 30, 40, or even 50$ of the applicants to not pass the test on 
the first attempt; despite these passage rates, the training institutions are 
rarely blamed. The relationship in those circumstances is between the testing 
agency and the prospective professional. Typically, the training institution will 
offer programs to help students who are having trouble passing those tests prepare 
to do better on the next try. A similar attitude with respect to the teaching 
profession is necessary and desirable. The more information that can be obtained 
on teacher testing programs, the better teacher education programs will be able to 
prepare to meet the needs of teachers in this regard. 

o Continuity of data gathering . We emphasize strongly that most of our 

recommendations for data-gathering will have little consequence if data are not 
gathered on a continuing and systematic basis over long periods of time. 
Frequently, what is needed is trend data, not individual data for a given year, 
since of overwhelming concern to the education community is continuing quality and 
meeting future education needs with present teacher preparation. 



Sources of Data 



NCES gathers and organizes dita in three principal ways, according to Attachment B 
of the information provided in your request to us. First, a variety of data-gathering 
arrangements are in place in state education agencies. Second, the Center sponsors 
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voluntary response to sample surveys conducted by mail. Finally, NCES gathers data 
collected by other federal agencies. We have two general comments about these 
approaches to data collection. 



Voluntary Mail Surveys . Our observation of and experience with data collection in 
schools of education causes some concern regarding volunt *y response to sample surveys 
by mail. We assume, for example, that some, if not all, of the data concerning teacher 
supply are gathered from the complex questionnaires that are sent to selected registrars 
in colleges and universities around the country. Typically, these are passed on to 
deans of education for response. Unfortunately, many of the items requesting specific 
information require a response that is often too detailed and too complex for the 
respondent. Consequently , the questionnaires may be discarded, or more significantly, 
estimates may be fabricated for the purpose of appearing to be in compliance with the 
request. Therefore, wo conclude that, given our experience, much of the data are 
suspect . 

We recommend that NCES develop a data-gathering strategy that brings the Center 
into closer and more intimate contact with the potential respondent through reliance on 
professional associations. Recognizing that this strategy can be very costly, it would 
be more acceptable to allow a higher margin of error than to leave questions of accuracy 
unanswered. Within the constraint of scarce resources, the Center should focus on the 
selection of a smaller sample and take the steps necessary to enlist institutional 
support dnd involvement with the data-gathering strategy. More than likely, this would 
require personal contact by either a contractor or Center staff, but we believe that 
such contact is necessary in order to ensure the necessary respondent involvement. Even 
though the error margin might be larger than the ideal, the representativeness of the 
data is likely to be more powerful. 

12 
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Collaboration , The documen' "Indicators of Education Status and Trends" (January 
1985) lists the variety of other federal agencies and departments with which the Center 
works in the development of data-gathering strategies. Such efforts are to be 
applauded, since the richness of the information i3 undoubtedly enhanced by it. There 
was no mention, however, about collaboration with and use of data and information from 
the broad variety of non-federal sources. In the area of elementary and secondary 
education, for example, such organizations as the National Education Association, the 
American Federation of Teachers, the Council of Chief State School Officers, the 
National Association of Secondary School Principals, and the American Association of 
School Administrators, as well as others, are continually gathering information about 
their enterprises. We recommend that the Center initiate long-term collaborative 
relationships with these groups that would allow for an outlet for the important data 
which they collect, and also ultimately an increasing standardization of data that are 
collected by them. With the professional expertise a^d the broad-based access to data 
needs that NC£S possesses, it could be helpful in aiding organizations to focus their 
limited data-gathering strategies. This influence would lead to a richer national data 
pool that would provide practitioners with more, and more accurate, information, ana 
would also help reduce the number of problems that are encountered when one set of data 
appears to be contradicting another set. We offer no master plan concerning how these 
long-term collaborative relationships might evolve, but we remain convinced that such 
efforts would be worthwhile. 

Obviously, the same idea concerning collaborative relationships can and should 
apply to the gathering of data beyond the elementary and secondary education program. 
These types of relationships could also be developed in the areas of post-secondary 
education, vocational education, and education of a variety of special programs and 
populations that are of interest to NCES and to America's educators. 

Va 
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Current NCSS Data-Gathering Efforts 



Three aspects of the current elementary and secondary program of particular concern 
to teacher education are discussed below in detail. With respect to the State Aggregate 
Fiscal Report and the Public School Survey, we believe that additional useful 
information could be obtained via these instruments than is currently be^ng collected. 
With respect to the Survey cf Teacher Demand and Shortage, a revised data-collection 
methodology is suggested. 

State Aggregate Fiscal Report . NCES is committed to gathering state aggregate 
fiscal data concerning current expenditures by major function. Suggested examples are 
instruction, support services, and non-instructional services; it is unclear whether 
there are more categories than those. A category system should be added that allows 
data to be gathered concerning the state aggregate commitment to teacher education. 
This could include expenditures for inservice education, beginning teacher induction 
programs, and support for intern teachers as well as preservice teacher education. 
There is precedent for public school monies being used to support all of these 
classifications . 

Public School Surve y. It is important for the education community to know about 
the nature of teacher training beyond preservice teacher education. Do teachers take 
college courses as the primary strategy for inservice education? Do they enroll 
primarily in district-sponsored inservice education? Do they enroll primarily in 
inservice provided by their teacher organization or by other professional organizations? 
Are there more informal types of inservice training that teachers use? In addition, 
perhaps through the data provided by the ( 3tricts, .it would be helpful to find out the 
amount of money devoted by districts to inservice education as a proportion of their 
total budget. It is important to know the types of structures that school districts 
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organize in order to provide additional training for teachers. With the current thrust 
toward helping beginning teachers adjust to the classroom, it is important to start 
gathering data about the fiscal support for new teachers (induction programs). Finally, 
it would be very helpful if the Center could develop strategies for gathering data 
concerning the nature and frequency of relationships that local school districts have 
with institutions of higher education, focusing on inservice education. 

Survey of Teacher Demand and Shortage . With regard to the strategy for obtaining 
information on teacher demand and shortage, we reiterate the invocation for the more 
intensive sampling notion we presented earlier. In the current plan, 3540 educational 
institutions, 2540 LEAs, and 1000 private schools are asked to provide information on 
the number of budgeted teaching positions, number of vancanoies, etc. Wo have two 
concerns about this strategy. Fir3t, when over 7000 different people are asked to 
provide this kind of information, the return rate must be questioned, i.e., how many 
people do, in fact, respond tc these surveys? Second, the completeness and accuracy of 
the information provided is of concern. When questionnaires come across a respondent's 
desk, typically from sources toward which no affiliation is felt, the tendency is to 
complete them as quickly as possible, often with little regard for the thoroughness and 
accuracy of the information provided. How does the Center ensure these data are 
representative? Requesting the same information from a much smaller sample via personal 
contact, such as a phone interview, would allow surveyors to quiz people and make 
judgments as to how well prepared the respondent is to d^al with the question. 

Other Data-Gathering Efforts 

Data needs that do not appear to be adequately met by the current elementary and 
secondary program are described below. Teacher supply, teacher competency, and teacher 
career patterns are all areas where NCES could be an invaluable source of detailed data. 
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Teacher Supply . While no reference was found in these materials to teacher supply, 
and while it may be that the "supply" focus is part of the postsecondary program, it is 
important to mention it here* Currently, teacher supply estimates a^e typically made 
from data provided by institutions of higher education. More accurate sources of 
teacher supply data are stale education departments: the number of teachers certified 
and/or licensed, the number of emergency or provisional certificates granted, the number 
of applications that were not granted for one reason or another, and the number of 
requests from schoo] superintendents for special consideration in employing education 
professionals. State education department licensure figures, gathered over time, will 
provide trend lines on teacher supply that are much more unable than the data provided 
by IHEs. The reason for this is that many teacher education program graduates have no 
immediate goal of becoming classroom teachers, i.e., a remarkably constant number go on 
to graduate school, choose to stay out of the job market for reasons of marriage or 
family, or find alternate employment that is more appealing to them at that moment than 
teaching. Another confounding phenomenon is that the number of education degrees 
awarded does not equal the number of students newly certified to teach. Secondary 
education teacher candidates, for example, may have a degree in their subject area major 
rather than in education, and prior graduates can return for postgraduate work to obtain 
a teaching certificate. Thus, tha number of undergraduate degrees awarded from a 
school, college, or department of education gives only that — an estimate of the number 
of people who have undergraduate education degrees; this number may be quite different, 
across states and across institutions, from the number of those who are actually 
available to teach. We recommend that NCES take an active role in collecting data about 
teacher sv oly at the state level. 

Teacher Testing . NCES has a significant role to play in gathering data about the 
rapidly growing teacher testing movement in American education It is important for the 
education community to know which tests are used and the frequency of their use. 
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Although not necessarily NCES's responsibility, there should be data provided concerning 
the validity and reliability of these tests. To the extent possible, aggregate scores 
by region, type of teacher, etc., should oe made available. Data are needed concerning 
the relationship of teacher testing scores to job placement. It is also important to 
discover the level of discrimination that tests promote, i.e., do all teachers who take 
the test end up passing it or are some actually barred entrance to the profession? We 
recommend that NCES develop data-gathering strategies to address questions of the impact 
of teacher testing, in such areas as the competence of beginning and re-entry teachers, 
recruitment and retention of minority teachers, and the overall quality of education. 
Although some of these questions may go boyond the mission of NCES, we believe it is 
important that the questions be considered as an important context for development of 
longitudinal data collection efforts. 

Cci'tif ication and Licensure . More information is needed about certification and 
licensure requirements in the states. It is very difficult, at the current time, to 
know whether a license or certificate in one state has any relationship to that offered 
in another state. Reciprocity is decreasing, partly because of differences in testing 
programs among the various states and partly because of growing skepticism about program 
approval and program quality. More intensive analyses are needed of what stands behind 
certification and licensure requirements both across and within the various states. 

We recommend that NCES play a significant role in gathering data to increase oar 
understanding of the process of certification and licensure, a process which greatly 
affects the country's supply of teachers. Cooperation with the National Association of 
State Directors of Teacher Education and Certification could produce a ready supply of 
such data. 
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Beginning Teachers , Specific information from first- and perhaps second-year 
teachers over a ten-year period „ould be very useful for planning purposes. Knowledge 
about their route of entry — traditional preservice program, alternate program, lateral 
entry, and the type of certificate held — would be enormously helpful to policy makers. 
The type of support available to them as they began their teaching careers and 
information concerning experiences encountered in searching for a first teaching job are 
specific questions of concern: How many districts did they apply to? How many 
interviews did they receive? How many job offers did they receive? How far away from 
their home or college did they have to go to find a position? How far away ^rom their 
first choice did they have to go? The recent spate of literature concerning the 
importance of the first two years of teaching to a teacher's career adds validity to 
these questions. 

Teacher Development , By virtue of the current line of thought that teachers go 
through a variety of developmental phases, we recommend that the National Center for 
Education Statistics gather demographic, inser^ice education, and other data from 
teachers within the framework of their years of service. Such longitudinal data would 
help to answer important questions about degree of teacher retention, reasons for 
leaving teaching, reasons for remaining in teaching, development of professional 
competence, and impact of state and federal education initiatives in this area. 

Career Ladders/Lattices . Information about career options within the teaching 
ranks for teachers is important, given the strong movement in elementary and secondary 
school systems to provide more variety and reward in teachers 1 work. Typically, these 
are referred to as career ladders or career lattices. The education community needs to 
know what is being done acros3 states and what is the impact of these programs on 
teacher satisfaction, teacher retention, and the development of teacher competence. 
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Teacher Retention , The question of teacher retention has awakened considerable 
interest over the last five years. Although it may be possible, in a post hoc manner, 
to analyze some of the data gathered by NCES on this topic, it is crucial to insure that 
we have accurate answers to such questions as: How long do teachers stay in the 
teaching ranks? What are the reasons that prompt them to leave? What proportion of 
teachers opt or will opt for early retirement programs? What number leave and return at 
later stages of their lives? We recommend that NCES make this a focus for the data 
collection program. 

Early Retirement . Another specific category of teacher demand data that would be 
useful concerns the prevalence of early retirement systems in schools and estimates of 
what proportion of teachers are taking advantage of early retirement programs. A 
retirement age of 65 can no longer be assumed; many states and school districts are 
providing incentives for people to retire early and the large number of "baby boom" era 
teachers ar? reaching an early retirement age. There would be significant benefits from 
estimates of future vacancies as well as froiu the data that the Center obtains 
concerning real current vacancies. This kind of information would enhance the ability 
of teacher educators to make long-term program development decisions. 

Comment on "The Sorry State of Education Statistics" by Cooke, Ginsburg , and Smith 

Cooke, Ginsburg and Smitn state that education statistics as collected and 
published in the U.S. today are inadequate, inconsistent, incomparable, ?nd sometimes 
just plain wrong. They advocate a set of "indicators" which would standardize 
definitions, collection parameters, and interpretations across state lines. Wnile this 
is a neat theoretical solution, there are serious problems when it comes to application. 
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It appears that tne authors are highlighting a problem that 13 not necessarily 
related to "bad" data collection, but to uncoordinated and non-structured data 
collection. Frequently, those who wish to make points using data from a variety of 
sources are not sufficiently careful to document the shortcomings of the information 
and, just as frequently, they misinterpret the data by being insufficiently specific 
about the definitions. In one example, the New Jersey and Virginia vocational education 
enrollment problem cited in the paper is obviously a case where the count is of 
headcounts of students in classes, not number of people enrolled in vocational 
education. If that is made clear, the data make sense, thou^j they raighv not be 
nelpful. In another example, the problem presented by the authors in understanding 
dropout rates appears to be a situation where the term ' dropout" was probably defined 
differently by the census gatherers and by the school people. The authors point out 
reasons why people might intentionally falsify data, and that might be true, but the 
differences are more likely related to lack of definitional consistency. 



Cooke, Ginsburg and Smith propose common definitions of indicators across states. 
We would agree with that, but only to the extent that that would allow either a state or 
NCES to have a standard by which to compare state data. Thus, when a state chooses to 
gather data in a different way, or chooses to gather very limited amounts of data, the 
statement could be made, "Their data do not fit these indicators, thus we cannot use 
them." In that way Secretary Bell's chart would have omitted the states of Wisconsin, 
Iowa, and perhaps others, because so few children took the SAT or the ACT in those 
states. 



The paper points out that we're a very independent and autonomous country and we 
gather data in a variety of different ways. What the authors seem to be asking for is 
some system that will standardize the way terms are defined and data are collected. 
AACTE would submit that that is close to the collaborative relationship we have 

20 

ERIC 650 



suggested. It would be unrealistic, however, as well as inappropriate, to suggest that 
everyone collect data in a form determined by NCE5, Rather, it is likely that there 
will have to be careful separation of data with definitions different from these which 
the Center can certify as being standard and comparable. It would als° be helpful for 
the Center to offer a data-gathering methodology critique service where they will read 
over any state's (or other entity's) plan for gathering informacion and offer advice 
about improvement. This strategy would be even more helpful if the Center could provide 
resources to help states formulate better data-gathering methods. 

Summary 

The following statements summarize our recommendations for the NC£S ten-year plan 
for data-gathering from the elementary and secondary education sectors: 

1. The priority data needs of the teacher education community are for information m 
four areas: (1) teacher supply and demand; (2) beginning teacher induction programs 
and irservice education; (3) teacher testing; and (4) continuity of data-gathering. 

2. Collaborative relationships need to be established with professional associations 
and other groups with knowledge about the population being studied, and these 
relationships should be ongoing. 

3. Current NCES data-gathering efforts should be broadened to include the areas of 
financial commitment to teacher education and the nature of inservice education. 

4. The research design of very large samples should be modified to insure completeness 
and accuracy of responses. 
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5. Data should be collected on a continuing and systematic ba. over long periods of 
Lime. 

6. Additional categories for data collection should include: (1) teacher supply data 
at the state level; (2) the impact of teacher testing on recruitment, retention, and 
educational Quality; (3) the nature of certification and licensure; (4) the 
experience of beginning teachers; (5) teacher development via analysis by years of 
service; (6) the prevalence and nature of career ladders; (7) teacher retention; and 
(8; early retirement. 
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AMERICAN LIBRARV ASSOCIATION 



SO EAST HURON STREET 



CHICAGO ILLINOIS 60611 




October 3, 1985 



Emerson Ell iott, 
Administrator 

National Center for Education Statistics 
1200 19th Street, N.W. 
Washington, DC. 20208-1401 

Dear Emerson Ell iott: 

Please forgive ALA for not responding earlier to your letter of May 17, 1985 
addressed to Robert Wedgeworth. for. Wedgeworth left ALA in August to become 
Dean of the College of Library Science at Columbia University. During the 
last few months a number of important things were neglected, your letter among 
them, We sincerely regret that ALA has not taken part in the redesign of the 
NCES data collection program for elementary and secondary education. 

As you know ALA has been working actively with NCES to improve the collection 
and publication of statistics about libraries. Our Office for Research 
completed a contract for NCES in November, 1984 with a report entitled 
"Analysis of Library Data Collection and Development of Plans for the 
Future." This report recommended revisions in the forms used to collect 
statistics from College and University Libraries and from School Library Media 
Centers. We have been pleased to learn that both of those forms are being 
used this fall in much needed surveys. Robert Wedgeworth, ALA Executive 
Director, and Jo An Segal, Executive Director of ALA's Association of College 
and Research Libraries sent a letter to the directors of all college and 
university libraries urging them to complete the form promptly and 
completely. ALA's American Association of School Librarians is eager to see 
the results of the Fall 1985 Survey of School Library Media Centers as the 
data will be extremely useful to the committee engaged in drafting revised 
standards for school library media centers. Finally, the ALA Office for 
Research has just begun work on a contract, funded jointly by NCES and the 
Division of Library Programs, to conduct a pilot study leading to a 
cooperative system for public library data collection based on annua i data 
collection by the fifty states. 

As you can see ALA is very involved in the NCES data collect^ , efforts which 
relate tc libraries. School library statistics are of special concern to us 
because there is almost no other source of information on this topic. The OFR 
report noted earlier documents the fact that although some information about 
the other library types is available from states and other organizations, this 
is not true for school library statistics. We are very pleased that NCES is 
surveying school library media centers in Fall, 1985* The latest available 
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data was collected in 1978 and is no longer useful. We believe that 
statistics about school libraries should be an important part of any 
elementary and secondary school data collection effort. Data should be 
collected regularly on this topic and included in the compilations which 
describe other data collected on education—the of Education Statistics 

and/or the Condition of Education . 

We hope these ideas can be incorporated into your plan even though we have 
missed your deadlines. If there is anything ALA can do at tfns late date to 
provide additional information please contact us through Dr. Mary Jo Lynch, 
Director of the ALA Office for Research. 



Sincerely yours 



Roger"Parent, 
Acting Executive Director 

cc: Eileen Cooke, Director, ALA Washington Office 
Larry Larnour, NCES 

Mary Jo Lynch, Director, ALA Office for Research 




RP: Id 
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EDUCATIONAL STATISTICS FOR STUDIES OF POLICY AND ADMINISTRATION 



Robert G. Lehnen 
Professor of Public Affairs 
School of Public and Environmental Affairs 
Indianapolis r IN 46223 

Telephone: 317/264-3466 



(August r 1985) 



an invited paper prepared for the redesign of the elementary 
and secondary education data program of the United States 
Department of Education National Center for Education Statistics 
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Education Statistics for Studies of Policy and Administration 



I. Introduction 

The observations contained in this paper arise from the study 
Financing Indian a's Public fi£h££ls done in 1984 for the Indiana 
General Assembly 1 , The study used data from the National Center for 
Education Statistics (NCES) to compare Indiana with other states in 
chc areas of resources committed to education and in educational 
performance. The difficulties that arose in making these comparisons 
and the reactions of the supporter s and critics of the st udy are 
reported here. The remainder of the paper addresses some specific 

problems concerning NC^s statistics and practices and presents some 
recommendations for imp /ing them. 



II, Utility fif U£E£ Statistics £&L Policy Studies 

A central question facing most states today concerns the adequacy 
of their public education system, both in the areas of the resources 
committed to education and the performance of the system, Indiana is 
no exception in this regard. Having experienced severe economic 
hardships in many parts of the state, Indiana in recent years has 
taken a closer look at its public schools with the intent both to 
improve quality and also to make the state more competitive in its 
ability to attract industry and rptain its workforce. 

Financing Indiana's Pnblifi Sfillflfilfi was designed to review 
Indiana's position among the states and report on the effects of 
property tax reform, undertaken in 1973/ on its 304 school districts, 
NCES statistics played a central role in accomplishing the first 
purpose^, and statistical information ,pn2 from the Indiana Department 
of Education provided the basis for district by district comparisons 
of the effects of tax reform. The discussion in this section is 
confined to the role that NCES statistical information played in the 
Indiana report. 

The principal NCES measures used in the report fell into two 
categories: (a) measures of input (resources) and (b) measures of 
output (performance). The report reviewed the availability of 
education statistics that both measured, in some general way, one of 
these two concepts and also provided state by state comparisons. Most 
mea sures reported in NCES publications did not meet these two 
requirements, particularly the latter one. 

The measures eventually used in the study to compare Indiana to 
other states are as follows: 
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Input & 



Average days attended per pupil enrolled, 1975-76 
Number of pupils per teacher, 1980-81 

Number of pupils per teacher based on enrollment, 1980-81 
Total expenditures per pupil 

Total public school expenditures as a percent of personal 
ircome 



Output Measures 

Median years of education 

Percent graduating from high school 

Average SAT score (for 22 states) 



Some of these measures were not ideally suited for the task. The 
output measures often were criticized as being too vague and not 
reflecting performance, in particular, the lack of suitable measures 
of achievement and other aspects of educational performance limited 
the effectiveness of the study. 

Among the input measures Total Experditures per Pupil received 
the most attention. A series was constructed from data provided in 
tables reported in issues of the Digest q£ fidjiiiaiifin Statistics to 
show trends in national and state expenditures. This analysis 
received considerable comment from General Assembly members, the 
media, and various interest groups. Those critical of the conclusions 
of the study- -that Indiana was substantially behind other states in 
levels of spending for public education and was falling further 
behind — argued that the NCES data were unreliable and were not 
uniformly reported by the states. The critics thus concluded that 
Indiana was, in fact, better off than what the NCES statistics 
indicated, and the conclusions of the study must therefore be 
di. counted. These and other issues in the use of NCES statistics for 
making policy recommendations are discussed in more detail in the next 
sections. 



III. Improving ±h£ Utility q£ H£££ Statistics in PfiliCY Studies 

Perf crmance Measures ! One conclusion reached in doing the report 
for the General Assembly is that there are few good measures suited 
for policy and administration studies. One can divide policy and 
administration measures into three categories: input or resource 
measures, process or administration measures, and outcome or 
performance measures. Although some measures reported in the Digest 
flf Education statistics may be suitable to use for one of these three 
purposes, most fail on other accounts d'scussed below. 

The area where most attention i~J needed is on the performance 
side. The question most as! 4 *d by Indian:* General Assembly members was 
about the effects obtained from various programs and expenditures. 
How can one know that if spending is raised, or class size reduced, 
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that the schools will produce a better educational product? The three 
output measures used in the Indiana policy study were poorly suited to 
evaluate the performance of the Indiana educational system as a whole, 
and thus anecdotal information received as much credibility at times 
as national statistics. 

Less A ggregation : The current measures of education (input, 
process, or output) are not very useful because of their present level 
of aggregation. National averages and other statistics do not reveal 
much about the state educ tion systems. Since education is primarily 
a local and state function, it is essential to disaggregate the 
information to state and district levels. Without this detail NCES 
data will have only limited utility for policy studies within states. 
Yet it is the states who will determine the direction and scope of 
education policy and not the federal government. Without tMs detail 
NCES data will have only limited utility for policy studies among or 
within states. 

£fimpa£abili£¥ : it should be possible to obtain uniform 
information about every school district in the nation. Since most 
states including Indiana have their own departments for recording 
state and district level education statistics, one may argue that the 
state is the proper place to maintain such detail. If the states are 
to become the repository of state and district information, then the 
measures reported by the states end NCES must be the same. In the 
course of the Indiana school finance study, it was not possible to 
construct district-level measures of "national" statistics even though 
the information had come from Indiana. For example, the measure Total 
Expenditures per Pupil reported by NCES was not available by district 
in Indiana. 

L££j£ flf Dfl£lnnfiHtatifin : There appears to be no technical 
publication reporting NCES operational definitions, technical terms, 
standards, practices, and quality control. Early in the Indiana 
school finance study, a reference librarian at Indiana University 
attempted to obtain such a document without success. Subsequent calls 
to NCES and a conversation with a staff member revealed that no such 
publication presently exists. The lack of such documentation makes it 
impossible to provide information about the interpretation of the 
statistical information. Furthermore, it compromises the conclusions 
reached by analysts using NCES data, because critics often use 
anecdotal or hearsay information to refute conclusions. For example, 
critics of the Indiana study charged that Indiana's average 
expenditure figure was "too low" because book fees, paid for in 
Indiana by parents and not by tax monies, were not included in the 
Indiana statistic. This observation could never be verified or 
refuted. 

Hfidia fQX Reporting : Thf eight NCES measures used in the Indicia 
study came from the following sources: key punching of selected 
tabular information from various issues of the pjge gfc q£ Eflygflt XQU 
Statistics , and key punching of a table reported in USA Today from the 
January 6, 1984, news conference by Secretary of Education Terrell 
Bell. The latter source was subsequently verified six months later by 
obtaining a copy of the press release prepared by the Department. 
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Several comments are in order regarding the means by which this 
statistical information was obtained. First, statistical series, such 
as total expenditures per student, should be identified as such and 
reported as a scries. Presently, finding comparable tables in various 
issues of the £is££i appears to be the only direct method of 
identifying series. Second, data should be available through other 
media than press reports and publications. The data released to the 
media during the Bell press conference should have been available on 
floppy diskette, computer tape, and other such media* In general, 
Digest issues should appear on computer tape in a manner similar to 
that used by the Bureau of the Census to report its CPUntyrCity fiata 
BftfiJt information. With the advent of the professional computer, data 
should be readily available on floppy diskette as well as on public- 
use tapes. 

As a case in point, the Indiana Department of Education provided 
its entire 10-year database for 304 school districts on computer tape 
in Statistical Analysis System (SAS) file format. Such assistance 
greatly facilitated the access to information and the analysis of the 
issues. 

In the next section suggestions are presented for the improvement 
and/or expansion of educational statistics. This list is not designed 
to be comprehensive but rather reflects the accumulated experiences 
and opinions of the author. 



IV. ££m£ Specific Suggestions Concerning Education Statistics 

Bureau at Census Data : The Bureau of the Census presently 
provides data on population and housing characteristics by school 
district. This series is a important source of information and should 
be continued. Two observations pertain to the Indiana public-use 
tape. First, the release of data should occur sooner. Secondly, the 
accuracy of the data needs to be verified. In the latter situation, 
the Indiana public-use tape contained numerous errors, including 
omission of districts and the combination of similar-named districts, 
and thus v;«?s unusable for the study. 

Public Opinion toward Education ; Several polling organizations 
such as the Gallup Poll have conducted surveys of public attitudes 
toward education issues. Such surveys should be continued and 
coordinated through NCES. Specifically, a standard national survey of 
opinion should be conducted through NCES at least annually and a 
eries of standard indicators developed. In addition, NCES should 
have a research program whereby specific questions may be added to the 
core survey to measure current issues. For example, the impact of 
private schools such as the "Christian academies" on public schools 
could be explored. The survey data should be available in a timely 
manner on public-use tape or similar medium. Competitive research 
solicitations should be offered to select the principal investigator 
for each year f s special subject. 
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Survey £f Schoo l Personnel : No doubt one of the least documented 
areas of education is the state of the education personnel system, the 
teacher and administrative workforce. One reads about "burn-out", 
victimization, and disillusionment among teachers and administrators, 
but NCES provides little in the way to document these reports. An 
annual survey of school personnel, including at least teachers and 
administrators, is needed to measure the conditions in the workforce. 
Some topics to be included in the survey are measures of "burnout", 
perceptions of working conditions, reports on use of summer time, 
expectations about the future, crime and victimization in the 
workplace, out-of-pocket expenses incurred by faculty, and 
uncompensated job-related duties. 

Labor Relations Jnf ppmation t NCES nee^fc to report on the state 
of labor relations in the nation's schools. It should provide 
information on such labor-related characteristics as the degree and 
type of professional bargaining units, the number and duration of 
strikes, and the time to settlement of contract negotions. These data 
should be available on a district by district basis. One question 
that might be ultimately answered from such data is the relationship 
between labor relations (timely, amicable settlement vs. strife-ridden 
negotions) and performance. 

Health and fiutXiiion: Many people believe that the nutritional 
behavior of students, both before school and in the school cafeteria, 
is related to discipline and other performance issues in the 
classroom. School cafeterias vary greatly in the degree to which they 
offer nutritionally balanced meals as contrasted to ones high in 
carbohydrates and "empty calories". Some teachers have observed what 
they believe to be "carbohydrate highs" that may cause behavior 
problems in afternoon classes. 

NCES, possibly in cooperation with the Department of Health and 
Human Services, should develop a series of studies to determine the 
nutritional value of meals offered in school cafeterias and consumed 
by most students. In addition, the nutritional education and behavior 
of students should be explored, including breakfast-eating behavior, 
knowledge of nutritional issues, and choices make in the cafeteria, 
and this behavior should be related to educational performance and 
behavior. 

Class SiZifi f Si Teacher Load Information : The current measures of 
average class size reported in the Di gest <q£ Educatio n Statistics do 
not provide sufficient detail to be of much use. The averages 
reported for Indiana, for example, in no way reflect the personal 
experiences of this author or those of teachers he has consulted. 
One general argument made local ly is that special education classes 
skew the class size distribution and distort the mean, thus giving the 
impression of smaller than actual class sizes. NCES needs to develop 
information on the variation in class size by distict, subject, and 
grade. An example of class size data is given in Table 1. Other 
statistical information such as the median class size for each subject 
and percentile information should be developed from such data. 
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TABLE 1 
Examp le q£ £1sl££ £±Z£ IflMs 

viivux i/xuul x w t~ • nDv u vii w i. a. m w x * « 

Grade: 7 

Size of Class: 18 19 20 21 ... Median 



Subject 

English 

Mathematics 

Reading 

Science 

"ocial Studies 



table entries are number of classes of a given size for 
each subject 



TABLE 2 

Example fl£ Teacher Class Lead Measure 

School District: ABC School Corporation 
Level: Middle School 

* Number of Students: 120-129 130-139 140-49 ... Median 



Subject 

English 
Mathematics 
Reading 
Science 

Social Studies 



teachers who teach part-time or more than one subject 
should be pro-rated on an FTE basis 
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A related measure that should be developed is Teacher Class Load* 
This is a measure of how many students a teacher in a given subject 
instructs each school day. Such data should be available by district 
and subject* An example of such data is given in Table 2. The 
development of the above information should be done for core subjects 
initially and then expanded to include the entire cirriculunu 

Meagtireg flf Performance; As indicated in the first section of 
this paper, NCES does not report sufficient measures of performance* 
The general question of what kinds of measures to be collected should 
be explored with various public interest groups, policy makers, and 
education professionals* Undoubtedly, some measures of achievement in 
basic subjects are required, but the measures should not be confined 
to achievement measures* The selection of measures to be included in 
NCES reporting should be done by a public process reflecting the 
contributions of many diverse groups. Whatever measures are selected 
should be reported at least annually by state and district. Without 
such information the nation's policy makers cannot effectively 
evaluate the nation's schools and develop programs to remedy 
deficiencies. 

£fimp£ xablfi E£££ndi:tiii.£ and ££i£nii£ Eata : The lack of 
comparability between states poses serious problems for understanding 
that nature of school expenditures and revenues. Although sufficient 
detail exists within Indiana for its 304 school districts, attempting 
to compare Indiana's practices to other states is extremely difficult 
or impossible. NCES should take the lead in developing a model state 
data base and reporting system for district level data. Although such 
data may be collected and maintained at the state level, standard 
format public-use tapes from each state should be available. 

Expenditure and revenue measures should be the core indicators of 
such a system but other measures such as enrollment and performance 
measures should be considered as well. The separate states may take 
responsibility for collecting and reporting the information, whereas 
NCES may report statistics of primarily national interest. 



v. ssm& Concluding Observations 

The present condition of NCES statistics severely limits their 
utility for policy and administration studies. Although this paper 
has suggested several areas where improvement is desirable, it should 
be noted that some recommendations have special priority. The 
principal areas for improvement should concentrate on developing more 
useful measures of education performance; producing less aggregation 
of information by providing state and district level information; and 
finally instituting better documentation, quality control, and 
distribution of the product. These enhancements, more than any other, 
should improve the condition of national education statistics. 
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EQptnotes 



1 Robert G. Lehnen and Carlyn E. Johnson, Financing IndianaJja 
Public SfihfliilS: hR hRSilZ&±S. Ql th& P^Si and ££££!DID£Jld£LlLii2II£ Lqjl the. 
Future , Indianapolis, IN: Indiana University, School of Public and 
Environmental Affairs (1984) , 160 pages. 

2 See "Chapter 4: Achieving Quality Education in Indiana: What 
Level of Funding Is Required?" in Lehnen and Johnson, cp. £i£« 
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Houghton Mifflin Company 



One Beacon Street, Boston, Massachusetts 02108 School Division 

(617) 725-5000 Cable Houghton 



June 20, 1985 



Mr. Emerson J. Elliott 
Administrator 

National Center for Education Statistics 
400 Maryland Ave. SW 
Washington, D.C. 20202 



Dear Mr. Elliott: 



Thank you for your letter to Mr. Tovnsend Hoopes, President of the 
Association of American Publishers, inviting our industry to make suggestions 
for the planned NCES redesign of its elementary and secondary data program. 
The AAP views this as an exciting opportunity and has asked our Research 
Committee to respond. 

At the June meeting of our committee, we agreed to do this in stages: 
(a) by filing with you, by June 21, 1985, a written list of suggested changes 
in NCES data collections for your published reports; (b) informal discussion 
of these suggestions with staff members of NCES during our committee's 
planned visit to your offices on August 1, 1985; and (c) if it appears 
warranted, to follow these steps with further written communication by 
September 30, 1985, and/or participation in public hearings, as mentioned 
in your letter to Mr. Hoopes. The following is our committee's list of 
suggestions: 

1. Estimates of secondary school course enrollments are needed much 
more frequently. By this we mean the type data NCES has supplied 
Summary of Offerings and Enrollments in Pub lic Secondary Schools, 
1972-73 (NCES 76-150), Course Offerings, Enrollments"^ and Curriculum 
Practices in Public Secondary Schools! 1972-73 (NCES 77-153), and 
A Trend Study of High School Offerings and Enrollments: 1972-73 
and 1981-82 (NCES 84-224). This is a critical data need. Such 
market-size estimates comprise one of the most vital factors by 
which educational publishers decide whether to publish and how to 
publish instructional materials. Such data every 10 years is 
clearly not frequent enough. Dramatic changes occur in a decade. 
Elhi publishers need such data every 2 years. We submit that suf- 
ficiently reliable data can be collected through probability 
samples at a reasonable expenditure by the Government. Enrollment 
data should cover Grades 7-8 as well as Grades 9-12. Reasonably 
reliable sub-sample estimates should show enrollment variations 
for (a) course duration (full-year, one semester, etc.), (b) 
geographic distribution (state by state, or perhaps by the nine 
census regions), ana (c) public vs. private schools. The raw numbers 
of students enrolled in each course, plus their percent of all 
students in all grades, are the key data needed. 
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of the major course areas are likewise a critical need. This should 
cover Grades K-6, especially if you account for Grades 7-8 enrollment 
components of K-8 curricula under Item #1 above. Experts presume that 
100% of students are enrolled in Reading and Mathematics in Grades K-6, 
However, as we know, there is considerable variation, especially in the 
lower grades (as well as in Grades 7-8), in the proportions of students 
enrolled, at each grade level, % in other course areas such as: English/ 
Language Arts , Spelling, Handwriting, Science, Health, Social Studies , 
Computer Sciences, Music, Art, Foreign Language, etc. Yet there is 
currently no central data collection pinpointing the sizeable variations 
believed to exist in the percents of students, grade by grade, who take 
these subjects. This is a serious data gap. The same data specifications 
(especially concerning the duration and/or frequency with which such 
courses are taught) — and the same supporting arguments — outlined in 
Item //l above apply here. 

3. Grade by grade projections of total enrollments (of all students in all 
courses) for each level" K-12 are a similar vital and frequent need for 
long-range planning by publishers. Data should be organized like Table 
6 in the NCES volume, Projections of Education Statistics to 1990-91 , 
an important annual document. Past data (from Table 22 in the NCES 
annual Digest of Education Statistics , another important document) should 
extend back 10 years, and projected estimates should extend 10 years into 
the future. Such projections could be delivered in both of two frequencies: 

(a) Annually : Grade by grade, K-12, for nationwide total enrollments, 
as well as sub-sample breaks for (1) the nine census regions and 
(2) public vs. private schools, 

(b) Every 2-3 years : Grade by grade, K-12, for nationwide total 
enrollments, as well as sub-sample breaks for state by state, 

4. The number of units required for high school graduation in each of 

the arious course areas, state by state, are important data — needed 
annually. 

5. The specific courses (and their duration) mandated in high schools, 
state by state, are similarly important data — needed annually. 

Our Research Committee, Mr. Elliott, respectfully submits that "he above 
key data needs will enhance decision-making on a broad basis throughout the 
education community. More prudent decisions by publishers, large and small, as 
represented by the AAP, lead to a better choice of more competitive and suitable 
instructional materials for all school systems. 
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Please let either me or Ms. Barbara L. Meyers, Assistant Director, 
School Division, AAP, at its headquarters in New York know whether it will be 
convenient for our Research Committee to discuss these and perhaps other issues 
with you and NCES staff members at our scheduled meeting in your offices August 1 
(as arranged by her and Ms. Kay McKinney of NIE) . 




Sincerely, 



cc: Ms. Barbara L. Meyers, AAP 
Mr. Donald Ecklund, AAP 



36 

666 



THE ASSOCIATION OF TEACHER EDUCATORS 



THE IDENTIFICATION OF DATA NEEDED TO 
SUPPORT DELIBERATIONS ON POLICY ISSUES 



Submitted to the National Center for Education 
Statistics for consideration in the redesign 
of its elementary and secondary education 
data program. 



Prepared by Dr. Lee Bartolini 
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The Association of Teacher Educators, as an organization concerned about the 
improvement of elementary and secondary education, is keenly interested in 
the National Center for Education Statistics 1 plan tc redesign a 10 year 
program for elementary and secondary data collection. The organization is 
primarily interested, however, in data collection activities which will 
focus on the needs of teachers and teacher educators. In recent years, a 
flood of national and state reports have identified the need to improve the 
quality of teaching as a major educational issue. Data collection 
activities designed to provide the information regarding this issue would be 
most beneficial to decision and policy-makers. 

Some institutions, primarily state education agencies, have tried to 
identify and collect data needed to improve the quality of teaching and 
teachers. These efforts have included, but have not been restricted to, the 
components of teacher training programs. Additional information useful to 
decision-makers includes local district recruitment, selection, and 
evaluation procedures, and statewide teacher supply and demand statistics. 
These subjects are areas in which new data will be especially useful to 
decision-makers for elementary and secondary programs. 

The Council of Chief State School Officers (CCSSO) is also interested in 
knowing what states are doing to improve the quality of teaching. A 
committee of CCSSO on Teacher Education and School /Col lege Collaboration 
recently initiated a 50-state survey which attempts to identify those 
activities which have occurred or will occur relevant to four policy areas: 
attracting persons to the teaching profession, preparing persons for 
teaching, licensing persons for the teaching profession, and retaining 
teachers. The CCSSO survey also includes a component on teacher supply and 
demand. The policy areas identified by the CCSSO and the state education 
agencies provide a framework for identifying data needs. 

The NCES plan to redesign its data collection activities focuses upon 
elementary and secondary education. However, as suggested above, some data 
identified may be available from or through state education agencies or 
regional levels of government. NCES may want to consider these sources when 
planning data collection activities. Another consideration is that teacher 
education, historically, has been a state concern. Therefore, national data 
would be most useful if it could be generalized to specific states. 

Much information needed to make decisions regarding teaching is available 
only through colleges and universities. Information on the recruitment and 
selection of potential teacher candidates by institutions of higher 
education, information regarding the components of teacher education 
programs, and information concerning requirements for satisfactory 
completion of teacher preparation programs are examples of data needs which 
greatly affect elementary and secondary education, but which must be 
obtained through institutions of higher education. NCES also needs to 
consider this source in its data collection plan. 

Specific types of data relevant to the improvement of teaching in local 
schools are outlined in the following sections. Consistent with the scope 
of the NCES plan to redesign its data collection activities, the data needs 
identified focus primarily, but not exclusively, on elementary and secondary 
education. Data have been identified by posing a series of questions. 
These questions have previously been raised by policy-makers and serve as 
guidelines for identifying specific data which need to be obtained through 
q data collection activities. 
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A SAMPLE OF 
QUESTIONS TO BE USED '0 IDENTIFY 
DATA NEEDED TO SUPPORT DELIBERATIONS ON POLICY ISSUES 



I- Teacher Supply and Demand . Teacher demand is usually defined in terms 
or needed staff in soecTTrc subject areas within individual states. 
Colleges and universities, however, prepare teachers not only for the 
states where they are located, but for the rest of the nation as well. 

A. What is t!.e supply and demand balance for teachers by state, by 
region, and by specific subject area? 

B. In what states or regions is demand expected to increase? Decrease? 

C. How difficult is it for teachers prepared in one state to move to 
another state where need might be greater? 

1. What certification requirements are common to most states? 

2. What core of certification standards and qualifications would 
enable persons preparing to teach to meet most state 
requirements? 

3. How common are reciprocal agreements between states which 
would allow persons prepared in one state to teach in another? 

D. Can non-teacher experts (persons not prepared as teachers but 
recognized as experts in subject matter areas) be used as classroom 
teachers? What are the restrictions or limitations? 

II. Recruitment of Prospective Candidates into the Teaching Profession . 
Much has been said about the increased opportunities for women in the 
field of business and other professions, thus removing them as 
potential teacher candidates. There is also concern that the teaching 
profession does not attract the best and brightest students. 

A. Why do persons choose or not choose to become teachers? 

B. What local working conditions, salaries, or social conditions would 
make teaching more attractive to prospective candidates? 

C. What problems are associated with recruiting staff for 
extracurricular activities? 

Ill- Recruitment and Selection of Teachers by Local Districts . Local 
districts may use a number of techniques to recruit and/or select 
teachers. While some criteria are well established, such as a 
satisfactory academic record, little is known about the variety of 
criteria or degree of deference in the use of specific criteria. In 
addition, little is !.,iown of the factors which limit or constrain 
recruitment and selection practices. 

A. What procedures are used by districts to recruit teachers? What 
techniques have been particularly useful in identifying and 
recruiting quality candidates? 
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B. What do local districts do when qualified candidates cannot be 
recrui ted? 

C. What practices are employed to recruit minority candidates, in 
addition to routine recruitment practices? 

D. What factors most constrain recruitment of qualified personnel in 
local districts? 

E. Are district selection procedures clear and well defined? 

1. Do districts have fully developed job descriptions? 

2. What qualifications are required of all candidates? 

3. Do district qualifications for positions exceed minimum state 
requirements? 

4. Does the district (or state) require candidates to take 
aualifying tests? What tests? 

5. Who are the personnel who actively participate in the 
selection of educational personnel? 

6. What are the most important criteria used in the selection of 
teacher candidates? 

7. Whet does a district look for during an interview with a 
candidate? 

8. Does the district assess a candidate's writing skills or 
abilities? 

9. Do districts routinely select applicants who are certified to 
teach in more than one area? 

IV. Evaluation and Performance of Teachers . Evaluation practices 

implemented in local districts may have an important effect on the 
quality of education. More needs to be known about how staff 
evaluations are conducted and what practices, if any, are employed in 
staff development. There is also concern that budget restrictions and 
declining enrollments have caused school administrators to assign 
tenured staff to teach in fields for which they lack sufficient 
preparation. 

A. Do districts emplcy formal ^valuation procedures when assessing the 
performance of teachers? 

1. Are standardized evaluation instruments used? 

2. What specific criteria are used to evaluate staff? 

3. How often are staff evaluated? 
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B. Who, in local districts, evaluate teachers? What special training 
is required of the:e personnel? 

C. If an evaluation indi 

<- -' vv» ii ipp^iiimv i j i i >iiuiiuc id 

unsatisfactory, what subsequent action is taken? 

D. What programs for staff development currently exist in local 
school s? 

E. What restrictions or constraints hinder staff development or the 
improvement of performance? 

F. What relationship exists between performance on standardized 
teaching tests and performance in the classroom? 

G. What evidence is there to suggest that persons, either because of 
insufficient formal training or because of a long absence from a 
given teaching f-eld, are being asked to teach classes for which 
they are inappropriately prepared? 

Retaining Tea chers . Major concerns of those interested in improving 
tne quality w r education are retaining the best and brightest 
practicing teachers and removing the incompetent teacher. Conventional 
wisdoiu suggests that many of the best teachers leave the profession for 
positions in Drivate industry. Yet, little is known about those who 
leave. 

A. Wl'at is the attrition rate of teachers? Is teacher turnover 
greater or lejs than turnover of personnel in other professions? 

B. Are those who leave the teaching profession the most qualified 
teachers? The better performing teachers? 

C. Why do practicing teachers leave the profession? If they leave, do 
they ever return? 

D. What working conditions or approaches (e.g. merit pay, 
differentiated staffing, etc.) would act as incentives for keeping 
the best teachers in the classroom? 

1. What are the factors that provide the most job satisfaction 
for teachers? 

2. What are the factors that create the most dissatisfaction . jr 
teachers? 

E. What are the constraints associated with retaining the most 
competent teachers? 

F. What are the constreints associated with removing the incompetent 
teacher? 
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The Council of Chief State School Officers is pleased to have the 
opportunity to comment on t-he National Center for Education Statistics ten 
year plan for elementary and secondary data collection. The Council is a 
non-profit organization comprised of the state superintendents and 
commissioners of education in the fifty states, six extra-state 
jurisdictions and the District of Columbia. They are the executives 
responsible for administering the nation's public education enterprise 
consisting of approximately 85,000 schools with approximately 40,000,000 
students and an annual expenditure exceeding $119,000,000,000.00. When 
combined with postsecondary and continuing education, this enterprise 
requires 37. 8Z of state government expenditures, 42. U of local government 
expenditures, and combined with federal contributions (4.1Z of federal 
expenditures) total expenditures for education rank second only to 
National Defense and International Relations in terms of expenditures from 
all levels of government. 

It is natural, then, that the stewardship of this enterprise demands 
complete and accurate information for accountability to the public and 
legislative bodies, for the support of effective decision-making, and for 
the assessment of educational progress. Aa a result, the Chief State 
School Officers as collectors, processors, responders, and users of 
education statistics are in a unique position to provide insight into the 
implications of plans, changes and needs f r data and information about 
education. 
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The Council believes that the National Center for Education Statistics 
has a vital role in responding to educational needs in the following 
general areas: 

1. Contextual Parameters or descriptors that describe the 
educational enterprise 

2. Indicators of the health or status of education and its 
relationship to other countries 

3* Special longitudinal and other statistical surveys and systems 
that are practical only at the national or federr 1 level 

4. Assistance to state and local agencies in the design and 
operation of activities at the state and local level. 

An effective combination of these four areas will result in increased 

opportunity for new information to be generated by the Center, researchers 

using Center data, and by policymakers analyzing effect and impact 

change. 

It is important to pojiat. out at the onset that the degree to which 
these fcur responsibilities can be appropriately met is highly dependent 
upon the level of funding for the various activities. The Council at is 
November 1984 meeting stated that the U.S. Department of Education should 
"Request increased appropriations for assessment and evaluation efforts by 
five to six times the current level (8+ million per year) to make the 
capability comparable with national reporting in health, agriculture and 
other federal statistical tunctions. "A failure to accept the cost of 
producing, reporting, and analyzing statistical information, and the 
subsequent provision of funds to support this cost, will limit any real 
advance to piece-meal efforts with neglible improvements." 

41 
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1» Contextual Parameters or Descriptions 

Although a great deal of attention has recently been focused on the 
combination of data elements to assist in policy analysis, there will 
always be a need for basic education data. How many districts, schools, 
students, teachers, administrators, etc. are there? How much money is 
being spent? Data that define the size and the scope of the enterprise 
are essential and will continue to be needed by users. 

The Council and its Committee on Evaluation and Information Systems 
(CEIS) has had a long and consistent history of supporting the concept and 
implementation of a Common Core of Data that describes the system 
statistically. The philosophy of a federal-state cooperative data system 
to respond to federal data needs from state and local administrative 
records is one that has been pursued since 1961. The necessity of 
maintaining a cooperative approach is critical to the continued and 
enhanced ability of the fecex^l government to collect either voluntary or 
contracted data on a systematic basis. Consequently, we applaud the 
National Center for its approach in soliciting input from a broad variety 
of aud?eaces on a formal basis. We encourage the involvement of CCSSO and 
its Committee on Evaluation and Infonnatlion Systems at each stage of this 
process. 

CCSSO encourages the Center to cooperatively define those d«:ta 
elements that can be efficiently collected with universe information that 
will improve sairple selection procedures without unduly increasing 

45 

ERIC 675 



reporting burdens. This would include data on the school district 
universe and Che school universe. 



Standardization and coordination of data definitions at the federal 
level is a role that may be appropriate for NCES. This coordination, and 
the attendant acceptance of the development and distribution of 
glossaries, is necessary to promote the improved comparability of 
information. Additionally, the acceptance of this role would increase the 
confidence of data users that information in given formats would be 
available over time and not subject to changing program emphasis or 
approaches • 

The Council will gladly assist, through its CEIS as well as in other 
appropriate ways, in the identification of useful, necessary, or improved 
CCD data elements as well as suggesting elimination of those that hrve 
proven to be of little value. 

The Council looks to NCES as a provider of information relative to 
non-public schools as a basis for analyzing total educational 
information. Additionally, the establishment of comparable statistics 
abjut education in other countries would be most useful as states analyze 
their own data sets. 



Finally, the provision ot current information is a goal that NCES 
should constantly be striving to improve. The CCSSO recognizes the 
problems in collecting data and its irapacr. on the delay in publishing 
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information but is convinced that a rapid turnaround of information is 
essential for improved services. The recently established bulletin board 
is a positive step in this direction but electronic display of old 
information is holding out only half a promise for improvement* 

2. Indicators » 

The Council fully endorses the Condition of Education and the 
Indicators of Education Status and Trends and encourages continued 
cooperative development in conversion of statistical data into information 
that is useful in describing the effects of the schooling process. 
Appropriate statistics that are not part of the Common Core of Data should 
be gathered by NCES from other sources or through special surveys or 
procedures using sampling whenever possible* Analysis of the design of 
these special surveys or activities should consider the possibility of 
state use and in addition to the national requirements* The aggregation 
of data about education collected by other federal agencies (such as those 
reported in the Condition of Education and the Indicators ) info a common, 
accessible data base such as the newly created bulletin board, could be of 
considerable value to the states as data users* 

3. Special longitudinal and other statistical surveys and systems that 
are practical only at the national and federal level . 

The Council is fully supportive of the NCES High School and Beyond 
Survey and the planned National Education Longitudinal Survey of 1988. 
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The activities have proven to be extremely helpful in generating a variety 
of new information on courses, attitudes, relationships, and results. As 
stated earlier we would encourage consideration of developing these 
activities in a way that results in state representative as well as 
nationally representative data. It is recognized that this introduces 
additional cost for these activities but CCSSO looks to NCES to serve as 
the state's advocate in soliciting necessary funding to accomplish this. 

4. Assistance to State and Local Agencies 

A review of effective statistical and information systems clearly 
indicates that successful programs are dependent upon the capability of 
respondents to provide accurate information. Accurate and reliable 
information at the fedaral level is possible in direct proportion and 
relationship to the development anu improvement of support systems at the 
state and local level. This concept is embodied in the federal-state 
cooperative data collection systems which have involved direct federal 
financial assistance to states for the development of their systems. Such 
cooperative systems have become operational in the Department of Labor, 
Bureau of the Census, Law Enforcement Assistance Administration, and the 
Department of Agriculture. Other specific examples of such systems incude 
the Coopeative Health Statistics Systems and the Medicaid Management 
Information System. 

In education, however, assistance activities have had an uneven 
history and have been a woefully underfunded. As resources from all 
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levels become more scarce the competition for these limited funds by all 
aspects of general program administration will make it more difficult to 
develop improved data systems. Information and statistics are not a 
natural by-product produced at no cost; but rather a commodity which must 
be produced by someone and paid for by some agency. The CCSSO encourages 
NCES to seek sufficient resources to permit all levels of government to 
have resources to generate the data that the Federal Government needs to 
report timely, accurate and comprehensive statistics. If the resources 
are not made available, inadequate reporting results or funds and efforts 
must be diverted from more important activities related to program 
administration or instruction. Neither of those alternatives is 
acceptable. 

As indicated earlier, the Council would be most happy to assist in the 
detailed development of specific data items that make up the component 
parts of the ten-year plan in a more thorough manner. The need to analyze 
use of data collected, as well as the cost of data to provide are 
questions that need a great deal of attention and assistance from state 
and local providers and are as important as the definition of data 
elements. A federal data system that is not useful or practical to the 
state and local education agencies providing information will not 
succeed. Effective dialogue, coordination and assistance will allow an 
enhanced opportunity for all partners to access and use valid, reliable 
and timely statistics. 
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July 19, 1985 



Mr. Leslie Silverman 
Deputy Assistant Administrator 
Division of Statistical Services 
National Center for Education 

Statistic ^ 
U.S. Department of Education 
1200 Nineteenth Street, N.W. 
Washington, D.C. 20208-1401 



Dear Leslie: 



At our July 16, 1985 meeting of the CCSSO Ad Hoc Committee on the 
NCES Elementary-Secondary Data Program Redesign Project, it 
became apparent that the direction and limits of the project 
would be impacted by the perceived mission and functional 
boundaries assumed for the National Center for Education 
Statistics. We strongly urge that the function be a true 
statistical center that assumes the major responsibility for 
coordination of the collection, assembly, analysis and 
dissemination for that sector of society under its purview, 
namely education. 

The Secretary of Education would be required to make a clear and 
committed designation that the Center would have responsibility 
for coordination of statistical data collection and analysis 
activities across the Department of Education regardless of 
organizational lines and/or bureaucracies. Thic assignment would 
also require that the Center be charged with promoting the 
integration of the numerous data collection activities conducted 
by other federal agencies (Department of Agriculture, Bureau of 
the Census, Department of Labor, et al.) and related private 
agencies (National Education Association, American Council on 
Education, and the testing industry) to minimize burden on 
respondents and to develop increased standardization of 
terminology. 
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Mr. Leslie Silverman 
July 19, 1985 
Page 2 

This coordination role would include: 1) first and foremost, the 
coordination of the various activities currently under 
development in NCES (e.g., CED, VEDS, NELS-88); 2) expansion of 
the system to include those other data collection activities by 
the Department of Education (e.g., Special Education, Chapter I 
of ECIA, Chapter II of the Math and Science Act); and finally 3) 
establishment of out-reach activities to other agencies to ensure 
appropriate federal and national coordination. Included in this 
function would be defining a common set of data elements across 
the spectrum, coordinating collection of all statistical data, 
developing efficient collection and dissemination systems (in 
conjunction with users and providers), seeking out current needs 
for educational information, and providing assistance, both 
technical and financial, to the respondees and users of 
educational data. 

Any effort at a ten-year plan, without a clear understanding of 
the agency's mission and philosophy, offers little promise of 
success. Additionally, in our view, the failure to expand the 
mission and functional boundaries of the National Center to a 
true center for education statistics limits the potential growth 
co little more than that capacity which exists today. 




Staff, Council of Chief State 
School Officers 

GR:fkc 

cc: Emerson Elliott 
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September 20, 1985 
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Emerson Elliott 
Administrator 

National Center "or Education Statistics 
1200 19th Street, N.W. #606D 
Washington, D.C. 20208 
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The Council of Chief State School Officers f s Ad Hoc panel on the 
NCES Elementary /Secondary Education Redesign Project appreciates the 
opportunity you have afforded the task force to provide input into 
this important review and planning process. The recent heightened 
interest in educational statistics and information for program 
reform, system accountability, policymaking and applied research 
suggests the need for an accurate, timely and comprehensive data base 
of statistics aggregated in a manner that does not place an undue 
burden on current local and state information systems. We are 
encouraged that NCES has initiated such a thorough review and look 
forward to assisting the Center formulate alternative approaches to 
address this need. 

The Ad Hoc panel met on September 10th to review a draft of the 
M Synthesis of Invited Papers M and to consider future steps in the 
redesign project. This meeting resulted in the following general 
recommendations. 

1. The Center should develop a clear mission statement, along with 
an organizing theory for the integration of administrative record 
systems, sample surveys and longitudinal activities. The 
statement should address the Center's role in approving, 
coordinating, aggregating, maintaining and reporting information 
collected about education from other units of the Department of 
Education and other Federal agencies. 

2. The regional public hearings, which the Ad Hoc Committee 
supports as an effective means for fully involving many 
participants in the process of providing and using educational 
data, should be scheduled for the first quarter of 1986 rather 
than the last quarter of 1985. The delay would ue justified by 
the time required to fully involve all parties impacted by these 
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Emerson Elliott 
September 20, 1985 
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proceedings and supply them with appropriate information in 
advance, including drafts of the plan. 

3. NCES should consider state representative samples on all 
NCES -sponsored surveys and longitudinal study activities. 
Although this approach could result in increased costs to the 
federal government, recognition of the potential importance of 
resulting information would justify the expenditure. 

4. Any attempt to construct model state and local information 
systems should include an examination and analysis of the record 
of past efforts such as the Midwestern State Educational 
Information Project, the USOE Handbook Series, the Belmont 
Project, the Committee on Educational Data Systems Manual and 
others. 

5. NCES should exercise caution in balancing the legitimate 
desire of researchers and policymakers for detailed information 
with the cost and capability of institutions providing 
information. In its redesign project, NCES should consider 
factors such as the separation between research and statistics, 
state and local policymaking as contrasted with the federal role, 
and finally, the cost of information systems and their potential 
intrusion on the instructional process. 

6. NCES should approach collection of data directly from local 
agencies with caution. While this is appropriate at times for 
sample surveys, it intensifies and compounds extant problems of 
data definitions, comparability, reliability and potentially 
detracts from the possibility of developing administrative record 
systems that will meet a variety of needs. Additionally, 
appropriate federal/state/local protocol should be honored in 
intergovernmental communications. CCSSO has long recognized the 
need to collectivly work with the federal government to ensure 
that data collected is valid, useful and collected with a minimum 
of intrusion. The Committee on Evaluation and Information 
Systems (CZIS) continues to be an effective vehicle for 
accomplishing this task. 

Again, the Ad Hoc Committee appreciates the opportunity of 
providing input into the process and encourages tKe writing team for 
the plan, NCES staff, and yourself to call upon us for assistance. 
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ASSESSING THE EDUCATION STATISTICS INFORMATION NEEDS 
OF NON-SEA PUBLIC POLICY DECISION MAKERS 



Invited paper prepared by The Council of State Governments' 
Office of Information Services for the 
National Center for Education Statistics. 



by 

Mr. E. Norman Sims, Director 
Office of Information Services 

Dr. Deborah A. Gona, Coordinator 
Survey Research Services 



Ji'-e 1985 



Assessing the Education Statistics Information Needs of 
Non-State Education Agency Public Policy Decision flakers 



The last, half century in the United States brought a significant 
change to education policymaking as our system of public education 
moved from one with a "political" character to one with a 
profess 10 nal" character. This has been noted by Cremin who explains 



The scnools of a century aqo -- pa, ty-dominated , 
patronage-controlled, professionalism ignored -- were 
transformed over the next half century by a combination 
of businessmen, professionals, and a new breed of 
university-trained administrators. Their enemy was the 
political machine and the political boss. 1 

One outgrowth of this movement was the development of a distaste 
by the professional education community for all things political. 
As a result, the school profession began to maintain the "purity of 
its motives and values and the sinfulness of party activity and 
partisanship.'^ i n .he late 1950s , howe.-er, educators began to see 

u l™lt io7n aS l h J f ,° CUS ° f 3 di5Content generated by their clientele, 
in .he 1970s, this discontent had grown to the point that serious 
questions were being asked about the quality and quantity of 
professional educational services which seemed to be growing in 
expense J Most recently this concern has been expressed by the report 
of the National Commission on Edi-ational Excellence which" recommends 
a wide range of educational reforms c 0 halt the "risinq tide of 
mediocrity." a 

In part because of this discontent, there is now > movement at 
the policymaking level away from technical, pr fessional educators 
toward domination by political actors. I ci<f and economic 
conditions, which have in the past suppor -d .he image of 
professionalism and independence, have chanced. Educational 
policymaking has become more political than " tec hn ical . 4 

Accompanying this movement have been two other trends which are 
of equal or super mr importance: a shift of the primary political 
arena for education from the local level (the local education agency, 

links ' 2 5t3te 1eve1 ( usuall y the State Education Agency, or 
SEA)'; and a resurgence of the non-SfcA state decision makers, such 
as the state legislature and the governor, as major factors in the 
aevelopment of state educational policies. 6 

various forces are combining to cause these shifts. They 
include: judicial actions -- part,.ularly in the area of school 
nnance -- which have forced many state governments to reconsider 
fundamental educational policies formulated within SEAs and LE As ■ 
federal .nvolvement in educational finance and policymaking which 
also spu-red the development and expansion of the SEAs; and the 
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impact of movements to reduce taxes, particularly by those who felt 
that they carried an unnecessary local property tax burden. 7 

As educational policymaking ha^ moved from the local, technical- 
issue level to the state, public policy decision maker, an unfortunate 
schism developed at both the level of the user (between the 
professional educator and the political decision maker) and the 
provider (between information and pol icy centers serving legislators 
and executive branch agencies and those serving educators). 

Unfortunately, for those involved in improving state educational 
decision making by providing adequate information, this schism has more 
than historical importance and is nowhere more apparent than in the 
area of research into the information needs and uses of these state 
decision makers and the provision of information to them. Obviously 
the non-SEA political actors make im^rtant education decisions. But 
we know very little about the information they use, want or need in the 
process. 

The federal government, through the U.S. Department of Education 
and its National Institute of education and National Cent^ for 
Education Statistics, has made great strides in helping to improve 
decision making and use of information in SEAs and LEAs , not only 
through research into better education programs but also through the 
the dissemination of the results of these research efforts. Indeed, 
numerous studies have suDpl ied information about educational decision 
making within the SEA. 8 Moreover, work by a variety of educational 
groups has er.light'd us about several aspects of information delivery 
to decision makers within the SEAs and LEAs. 

The state Capacity Building (CBG) and Research and Dev el opme-t 
Utilization (RDU) projects, as well as the Research and Development 
Exchange (RDx) and National Diffusion network ( N DN ) programs, have 
provided crucial data about facets of the information process. 

Throughout the working life of each of these projects, new 
knowledge emerged about the process of decision making, resource 
delivery and client assessment. 9 Bjt as Mattas and Rawnsley have 
suggested, in the design and operation of information services it is 
important to know more than which members of the ed ucational community 
(the d irect cl i en tele of the educational research" .immunity) .r.ake use 
of services offered and what information they request. Research 
interest should also be directed tow* d aTl_ knowledgeable, and 
unknowledgeable, information users o„d non-users who have influence on 
the policymaf'ng process JO 

Again what is quite clear as we consider the great national 
debates over such issues as the role of the private sector in 
education and the operation of our intergovernmental system, is that 
the major policy decisions which affect education will not be made 
entirely in the SEAs and LEAs. They will be made by legislators, 
legislative staffers, governors, budget directors, state planning 
officers and others. Thes*- other actors may be familiar with the 
results of educational research and its application to policy, but it 
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is more likely, as we shall see later, that their knowledge is scanty 
(coming to them secondhand) and based upon information which is poorly 
provided or limited in applicability. 

It is also likely that educational decisions at the state level 
will be based upon information provided by sources other than those 
normally considered as information providers by the education 
community. The Council of State Governments 1984-85 edition of the 
Book of the States , for example, lists 90 organizations which may be 
caTTed upon by state policymakers to deliver this information. 

The day when, as political scientist Alan Rosenthal reports, a 
typical state legislator, asked about the legislative role in 
elementary and secondary education, replied quizzically, "Education is 
a local thing; we don't have anything to do with that; there's a 
formula", is over. 11 The schism between the technical and the 
political, which has led to research into the information needs and 
uses of professional educators while ignoring the political community, 
and the service agencies that support them, needs to be bridged. 

It is the view of The Council of State Governments that the U.S. 
Department of Education acting uniformly or through an internal 
entity such as the National Center for Education Statistics should 
take the steps necessary to construct this bridge by planning and 
taking action to achieve three program goals: 

— Increase our knowledge and understanding of how state 
educational policy decision makers use statistical information 
to make decisions and about their information needs; 

— Assess the capacity of statistical information providers to 
assist state education policymakers to make better use of 
available statistical information resources, and offer 
statistical information providers insight l n to mec hani sms 
for improving their services; and 

Based upon this information user and pro/ider analysis, 
develop a plan for improving the commun i :ati on of useful 
statistical information to the non-SEA state educational 
pol icymakers . 

When these goals are met, it is The Council's \ ">w that tne U.S. 
Department of Education will have significantly added to our knowledge 
of how the process of governing education and making policy decisions 
might be made more effective. It will have also increased our 
understanding of the nature of program administration by keying on 
executive and legislative branch decision makers as information users 
and the national service agencies (such ^s The Council of State 
Governments) which support them. 

It is the experience of The Council of State Governments that the 
provision of better information to state officials does result in 
better decision making. But information is a powerfuT~tool only when 
it is provided to the right people, in the right way, at the right 
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The effort we suggest would assist the education community in 
forging more powerful information tuois by providing the knowledge to 
give these tools better form and function. 



The Information Needs of State Policymakers 

In the early summer of 1971, a symposium was held in Arlington, 
Virginia, to review the experiences of various state and federal 
managers with the insti tutione 1 i za tion of federal programs at the 
1 ocal 1 evel . 

This symposium, sponsored by the MITRE Corporation, the National 
Institute of Education and the National Institute of Law Enforcement 
and Criminal Justice, focused on demonstration projects, but the 
comments made by the participants were telling from a number of 
perspectives. Mr. David L. Foote, Executive Director of the 
Colorado State Office of Planning and Budgeting, provided support 
for the effort we suggest when he told the federal program managers: 

If I had a single recommendation to make to the federal 
establishment ... it would be to take planning seriously. Not 
planning for demonstration, but what 1 would call policy 
planning, and making sure that we benefit from utilizing 
information that we continually generate and ask others to 
generate. ^ 

As NCES has indicated through this requst f or papers, the 
public policy challenges facing our system of federalism 
particularly in the area of education and its administration are 
formidable end can only be dealt with, as Mr. Foote indicates, by 
providing information to state policymakers in the most effective 
ways possible. But as we have noted above : these policymakers 
include a broad spectrum of public officials inside and outside of 
the SEA. 

In studying how legislative, administrative and judicial policies 
and governmental organizations affect education, the most important 
questions may be: What statistical information is used by state 
policymakers to make decisions?; What information do they think they 
lack to make better decisions?; How do they wish the material to be 
presented to make it most useful?; What lessons can statistical 
information producers and providers learn from the information needs 
and wants of these state policymakers? 

It seems, however, that because of the pol itical /technical 
education schism, educational research has not focused on the 
information needs of decision makers in the political environment. 
One portion of the effort The Council would propose to NCES would be 
to attempt to study the information barriers to good education policy 
decision making which exist in that environment. 
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A project recently completed by the ERIC Clearinghouse on 
Teacher Education attempted to overcome one of the barriers t( sound 
policy planning by identifying state policymaker information needs. 
But this study focused on three of the national state government 
service agencies rather than the policymakers themselves. 

Several efforts have been made by the state government research 
community including efforts by The Council of State Governments -- 
to assess the information needed and used by state public policy 
decision makers. They have dealt, for the most part, with the 
elected members of the legislative branch, but have not considered 
conditions as they specifically affect education. These studies have 
centered on: the peculiarities of the legislature as an infonnation- 
using institution; the kinds of general information legislators say 
they need; tne interpretation of their needs by their staff; certain 
effects of the decision-making process on information use; and 
sone constraints of the political environment on good information use. 
The findings of these non-education issue specific studies provide 
the research background for the first goal of the proposed effort. 

In general, state officials (particularly state legislators) must 
make many decisions within a relatively short period of time. This is 
largely due to the fact that the volume of state business with which 
they must deal is rapidly increasing and the questions put before them 
often require quite specific and detailed knowledge of the issue area. 
As a result, the kind of information these officials need in order to 
make timely decisions is often not readily available. 13 

Moreover, researchers have found that these time constraints have 
forced the state official to become an information "schizophrenic." 
Generally the decision maker says that he or she wants information 
that is trustworthy, objective, reliable, comprehensive, applicable, 
and timely. 14 However, while they might be quite vocal about the 
quality and comprehensiveness of the information desired, they rare]y 
seek elaborate information on policy issues. Indeed, in the state 
legislature this might lead to an information overload causing, 
"Paralysis (and) making things incomprehensible and unmanageable." 15 

As a result, although policymakers talk about their information 
needs, and on some topics their needs may be intense, they rarely 
seek elaborate information. When th*y get it unsolicited they do not 
know what to do with it, 15 Even legislative staff report a difficult 
time interpreting legislators' requests because of this duality of 
information needs. 1' 

If the literature's portrayal of state legislators (and, although 
most of the research tends to deal only with the legislative branch, 
there is reason to believe that these conditions exist within the 
top-levels of the executive branch as well) is accurate, why should we 
be concerned with the statistical information needs of these actors? 
When we consider the "general" use of information, it does seem as 
though this schizophrenic information-seeking behavior would argue 
against any efforts to improve the prov i s ion~of information relating to 
education issues. 
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But while this composite portrayal is essentially accurate, it 
is misleading when viewed apart from other aspects of the state 
decision making process. 

Tnroughoui a legislative session, fur example, legislators will 
receive information they cannot use. This is because the 
information did not reach them at the right time in the legislative 
process; it did not help with problem solving; it did not support 
the decision makers 1 predispositions; it was not in a format which 
enabled the legislators to relate it to constituent needs; or it 

did not tell the legislators how colleagues felt about the 
issue . 18, 19,20 

It is also a political reality that not every legislator or 
executive branch decision maker will be interested in every issue. 
Yet these decision makers will still face the prospect of having to 
make decisions on those issues either by endorsing them as executive 
policy or voting for or against them during the legislative session. 

As a result, these officials will be forced to rely on decision 
making "shortcuts" in order to survive the flood of decisions that must 
be made. One method legislators have for making quick, but palatable 
decisions, is to rely on the orientations or predispositions they have 
brought to, or developed early in, their legislative careers. It is 
improbable that the individual legislator will have preset notions on 
every issue to be addressed, but it is likely that one or more of the 
legislator's colleagues will have some ideas about, or expertise in, a 
particular issue area. It is expected then--and supported by the 
research literature—that when an individual policymaker cannot arrive 
at a decision on the basis of personal judgment, he or she will look 
elsewhere for assistance. But to whom does he or she look? 

Legislators look mainly to their colleagues and rely on the i r 
judgments. 21 A small group of individuals within the legislature "is 
likely to be regarded as expert in a particular subject or issue area. 
Other members can usually rely on their ability to produce policies 
which reflect the values of the group as a whole. 22 Various studies 
have shown that policymakers tend to ^ook within theV own group for 
cues for decision making rather than to outsiders (such as SEA, federal 
education, or educational lab and center staff). 23 , 24 , 25 , 25 Lobbyists 
particularly have recognized the importance of seeking out those 
members of the policymaking group who are seen as opinion leaders. 2 ' 
Sone education lobbyists have documented the^r strategies for informing 
these key legislators. 28 

These opinion leaders are also important in that they tend to 
occupy key positions in a two-step flow of communication between 
interested groups and individuals on the outside and the rest of their 
colleagues on the inside. 29 As a result, the ways in which these 
individuals with substantive knowledge make decisions, and the criteria 
they employ, will differ from that of their less knowledgeable, less 
i nterested , fel 1 ows .30 

But it is unclear just how these non-SEA "opinion leaders" make 
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their decisions, and what their statistical information needs are. 
More specifically, it is not clear how the non-SEA legislative and 
executive branch education opinion leaders make such decisions and 
employ statistical information. 

On the basis of recent studies^, we have some ideas as to why 
legislators become involved in education policy and are regarded as 
leaders in the area (e.g., legislator's background and/or district 
characteristics). On the basis of some isolated case studies, we also 
have some understanding of the factors which affect education 
policymaking in various states. 32 But these findings leave many 
questions unanswered and complicate our understanding of the decision 
making process as it relates to education: 

--The literature suggests that education policy questions are 
handled differently than other policy questions by state 
legislatures, simply because of the nature of the political and 
educati onal env ironment .33 

--The key opinion leaders in the field of education may not be 
make up solely of members of the education committees and 
education legislative staff, although that is usually where 
information studies related to education policymaking focus 
their attention. The leadership of the appropriations and 
finance committees, for instance, also have power over 
educational policymaking. A participant in education politics 
from a midwestern state notes, "They can have a nice time in 
house education talking about textbook selection, competency 
based education, and a lot of other things like that. Not much 
is going to happen on those things. It's the people who control 
the money who are calling the shots up and down the line. "34 

--The greatest source of information used by the state education 
community, the information produced by the federal government, 
is not widely used by the non-SEA public pol icymakers. Research 
on the influence of this information upon state political actors 
is surprisingly sparse. Thir is partly because it is relatively 
new, partly because the information is seen as beir.g more useful 
to the school community, and partly because the federal 
information base on educational operations has not resulted in 
an> theory-guided research literature,^ Indeed, one study 
found that federal information providers constitute th2 last 
group state non-SEA problem solvers call upon for solutions. 3 & 
Wirt has noted that the federal thrust for providing information 
to this group has been diffused, in part because, "Washington 
seeks to deal with complex organizations with a limited 
understanding of them. "37 

--The factor which may have the greatest effect on good 

information sharing between the education community and the 
public policy decision maker is the previously mentioned 
antagonism between the professional educator and the politician. 
Hal per i n notes that the schism between the technical and 
political actors continues with educators commonly saying that: 
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"Politicans have a short term view of the world; their primary 
interest is in their own constituency and their narrow 
sectional, ethnic, regional or economic interests; and are 
poorly informed on educational issues." The politicians, on the 
other hand, say: "In order to frame social policy, we need 
facts, not generalities. We also need practical responses to 
immediate problems. Yet rarely do educators have the 
information we need to make sound policy; and, educators ought 
to know how to communicate, but there are few groups that speak 
less clearly, less concisely, and with more obfuscatioti. 
Instead of precise, comprehensible, here-and-now language, what 
we get is usually too olympian, too Utopian, too abstract, or 
too fuzzy to be helpful." 3 ^ 



The Research Opportunity 

What is clear, after a brief review of the literature concerning 
the use of and need for information by state public policy decision 
makers, is that the research community cannot now identify mechanisms 
for providing these oolitical actors with more useful information on 
educational issues until more is known about the information 
env ironment. 

As we have noted, studies of .he use of information by the 
educational research community have historically centered on the 
policymakers in the state and local education agencies. Studies of the 
information environment of the state political actors have addressed 
general questions and have tended to favor studies of the legislature. 
These latter studies are even more limited—from an educational 
policymaking perspective—when we remember that the literature suggests 
that education policy questions are handled differently than other 
policy questions by state legislatures. 

Additionally, recent studies in the education area (such as those 
by Rosenthal) have centered on only a portion of the political 
information network: the legislative education committee members and 
staff. The literature suggests that two-thirds of this network may 
have been overlooked: the decision makers who affect educational policy 
but are not normally seen as being part of the system (such as 
executive branch planning and budget officers and legislative branch 
finance committee chairpersons), and the education opinion leaders in 
both branches. 

While informational barriers to better, more effective, state 
decision making exist in the political community, there is no evidence 
that these barriers are inherent to our political system. Barriers to 
providing adequate information for proper decision making were also 
noted in SEAs and LEAs , but have been greatly reduced by research 
efforts which have analyzed these barriers and suggested methods to 
overcome them. 

What remains is the problem of identifying the best means of 
providing useful statistical information to the political decision 
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makers within their unique environment. Given that information is 
"that which reduces error," we should conclude that pr iding better, 
more useable, statistical information to these important political 

tnrC Will Dnrnnrano imn^Awn/^ nrlii^^f iam^I **** 1 i — >. j ^ » • - _ j. _ _ j . r- 
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the SEAs, in the same way that providing better information to SEA and 
LEA leaders has improved their capacity to make better decisions. 

The research opportunity presented to NCES, then, is to: assess 
the information environment of the non-SEA education policymakers; 
review the results of this assessment; based upon this analysis, 
construct improved communication mechanisms to overcome the problems of 
legislative timing, relevance, personal predisposition, format, 
relationship to constituent needs and peer thinking; test these 
mechanisms; and disseminate successful approaches to the state 
educational policymaking community and those who serve them. 



Concl usion 

In its 1984 study of state response to the recommendations of the 
National Commission on Excellence in Education, The Council of State 
Governments found that state school leaders were taking the matter of 
reform for educational excel 1 ence to heart well before the Commission's 
report was finalized. 39 j n this regard the Commission's work « end 
the work of many other national taskforces assembled since the 
Commission's report was released serves as an additional propelling 
force for improvement efforts, but is probably not the initiator of the 
vast majority of state educational improvement efforts. 

If America is at risk, and if the reforms outlined by the 
Commission are what are needed to acUeve excellence, then there is 
every indication that the states are already taking the necessary 
actions. 

But what was also clear from ,r,e Council's study is that state 
officials are attempting to look ell beyond the Commission's report. 
Many states did not taice the Comr. >ssion's report for granted and 
numerous independent state and If. al tr^k forces have been established 
to identify specific areas for movement in each unique jurisdiction. 

It is well that they havf what seems clear is that for 

educational leaders to make nal headway in improvement efforts, 

additional experience is ne-. . communicating their "message" to the 
state political decision makers. The Commission gave weight to the 
importance of political actors at all levels of government, but is 
silent on the matter of how o^ ortunities become programs and needs 
become budgets. There was every indication from The Council's study 
that the channels of communication and the quality and quantity of 
information carried by them which bring about informed educational 
opinion in state decision makers, need additional study. 

The program goals The Council suggests to NCES in this paper would 
be an ambitious undertaking, but in electing to take action in this 
area The Council expects that the Center would be able to: accumulate 
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extensive knowledge about statistical information as it is used by 
non-SEA executive and legislative policymakers in making decisions 
which effect education; assess the meaning of the findings as they 
relate to the information, dissemination and research programs of the 
educational research community, the federal education actors, state 
education agencies, local education agencies, and education public 
interest groups; and develop a model for improving the communication of 
statistical information to these actors. 

The Council of State Governments would look with enthusiasm toward 
a long-range plan developed by NCES which would investigate the areas 
out! ined above . 

There may be no greater waste than information which goes unused. 
The Council looks forward to working with NCES to insure the widest and 
best use of statistical information by state decision makers. 
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Considerations in Redesigning the Elementary 

and Secondary Data Program of 
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Reliable measures of educational status and trends are essential if we 
are to be able to monitor progress, understand change and set policy in 
education. That statement may seem a truism applicable to any aspect 
of the national interest but it is especially true in the educational 
realm. It is worth citing some of the reasons why this is the case. 

The problem of deciding what data to collect is especially acute 
in education for several reasons. The first is the decentralization in 
this country of both the sources of information and the structures of 
decision-making in education. A second reason is that educational 
issues ace intricately interwoven with and affected by a host of 
factors — economic, political, social, demography — in the society, 
so that th?re are few natural boundaries to the relevant sources of 
information we n>ay need to draw upon. Many non-educational agencies 
collect uata that are r a latod importantly to schooling but often they 
collect them In a form that precludes their easy incorporation into 
educational analyses . 
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Education's characteristics of dispersed control, varied 
record-keeping, diverse goals, and incompatible measures of results 
serve both to compound the data-gathering problem and to generate the 
tremendous need for a systematic, coordinated approach to a set of 
indicators of the health of the educational enterprise. Moreover, the 
diversity within the country offers the hope that reliable indicators, 
derived in comparable form from different parts of the country or from 
schools operating under contrasting circumstances, can pay big 
dividends in increased understanding -f what practices seem to work 
well or poorly, so that we have a chance of not only ^ercribing our 
educational health but also improving it. 

Among the key questions to be addressed are who will use the data 
for what policy purposes, and w'i c information elements do they need to 
do the job? Further questions are what agency or agencies sMuld 
collect the data, how should they collect it, and what roi. should NCES 
play in locating, assembling, reporting, and interpreting the 
information? 

Audien ces 

There are multiple audiences for data about education. Since 
educational decisions are made at state and local levels, the 
information needs at those levels clearly must be met. The "local" 
decision-makers, however, need data not only about their own state or 
district but also about the nation. They need educational status and 
trend data on a raulti-discrict , multi-state, national and international 
basis in order to compare their needs, efforts, and accomplishments 
with those of others. 
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Several groups must have national and international data because 
their responsibilities are national. They incl-de federal legislators 
and administrators, research people examining the factors that 
influence educational progress or such questions as the interaction 
between education and economic development, and business people whose 
prospects for both manpower and m?rkets are affected powerfully by what 
happens in education, b< t h in the U.S. and abroad. 

Tnere is, then, a set of needs for data at the district, state, 
national and international levels Lhat arise from the responsibilities 
of diverse groups of people. There is no way to distinguish the levels 
of aggregation of data needed by persons with broader or narrower 
geographic areas of responsibility: the broader picture provides an 
essential context for even tightly focussed local decisions. 

Finally, the media and the public have a vital stake in the 
condition of education — a critically important "need to know." This 
need goes beyond raw data to a need for analysis and interpretation. 
This latter requirement is a hard one for any agency to meet in a way 
that will be perceived as even-handed but one that is nonetheless 
essent ial . 

Need for A Program to Delineate Issues 
The kinds of data needed obviously depend on the kinds of issuer 
to be addressed. The delineation of the issues that the data sho uld 
illuminat e is a critical step and one that needs explicit attention . 
We need an intensive effort to develop a taxonomy of issues to be dealt 
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with that will in turn generate the data to be collected. Second, we 
need a system for determining priorities within the taxonomic 
categories. Finally, we need to provide for regular review and 
modification of the taxonomy and priority matrices. 

Collecting, analyzing, and reporting data is expansive, despite 
the powerful advantages offered by sampling of both respondents and 
data elements. There must always be trade-offs leading to the 
inclusion of some kinds of data and the exclusion of others, with the 
subsequent regrets when people find in perfect hindsight that some 
critical information is missing while information of other kinds is in 
oversupply. 

While we will never be able to anticipate all questions, we might 
well be able to do a more satisfactory job of it if the effort to 
anticipate were itself made explicit and systematic. We need a 
taxonomy of issues within which to classify the questions to be asked, 
which in turn will generate a list of the data we need. The existence 
of an explicit matrix, giving shape and structure to the issues to be 
examined, would help to focus attention on the important policy 
questions at the most critical time, i.e. before the design for data 
collection has been decided upon. 

A recommendatio n to NCES, then, is that a project be commissioned 
to develop a taxonomy of issues to be addressed by education data and 
related statistics. This project would be of greatest value to the 
present redesign effort if undertaken in the summer of 1985 . Such a 
project could involve searching the literature in education and in 
other fields, preparing a set of discussion papers, and convening a 
working group of knowledgeable people to develop and publish a proposed 
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taxonomy and priority system for widespread comment and fcr suggestions 
as to overlooked sources of data. The results would be of enormous 
value to the U.S. Department of Education in pursuing the redesign of 
its data program. The taxonomy would be subject to periodic revision 
and expansion as new questions are proposed and defined in future 
years. 



An Interim Working Set of Issues 

In the absence of the results of a specific effort to delineate 
issues, we may turn to useful statements now available to guide our 
thinking. An excellent example is the set of issues developed in 1983 
as a guide to the reformulation of the National Assessment of 
Educational Progress. Excerpts from the pertinent section appear 
below: 

Policy Issues NAEP 
Should be Able to Addresr 

It seems clear that NAEP must now we a wide 
audience with diverse needs. Criticism of 
NAEP in the past has underscored its failure 
to be responsive to policy needs (Wirtz & 
Lapointe, 1982; Milrod, 1980; Wiley, 1981; 
Sebring & Boruch, 1982). What are some of the 
isues that NAEP should focus on as it 
reorganizes to meet the challenges of the 
eighties? 

Among the variety of pressing issues, three 
general policy areas stand out which should be 
addressed by NAEP because they require 
reliable data on student competencies and 
achievement : student competencies as they 
relate to national concerns ; student 
achievement and attitudes as they relate to 
human resource needs ; and, student achievement 



Messick, S. , Beaton, A., & Lord, F. National Assessment of 
Educational Progress: A New Design for a New ^ra , Princeton, N.J.: 
Educational Testing Service, March, 1983, pp. 11-15. 
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as it relates to school effectiveness . In 
addressing these issues NAEP must not only be 
able to provide a national overview, but must 
also be relevant to state and local concerns 
— not for the purpose of needless comparisons 
among states or school districts but to assist 
individual states and localities in meeting 
their goals and objectives. 

National Concerns 

Since NAEP 1 s inception, the federal government 
has designed and implemented education 
policies to provide equal educational 
opportunity to all citizens and to assure that 
young adults would be able to contribute to 
society in terms of both productivity and 
participation in the democratic process. The 
government clearly understands that an 
educated populace is a fundamental requirement 
for the nation's political and economic 
well-being. A major responsibility of NAEP 
should be to provide information for 
governmental and educational policymakers on 
the effects of their efforts and to act as an 
'"early warning system" of potential problems. 

At a minimum, NAEP data should be relevant 
to the following kinds of questions: 

Are today's students learning the skills 
necessary for productive functioning in 
America in the 1980s? The 1990s? The year 
2000? 

Are students in urban, suburban, and rural 
schools all being adequately prepared? 

Are public and private school children 
equally well prepared? 

Do children have access to programs 
preparing them to deal with the computer 
age? 

Are minority and disadvantaged youngsters 
being so prepared? 

What types of programs or allocations of 
resources seem to make a difference for 
disadvantaged and minority students? 

Are children from limited-English-speaking 
homes being provided the necessary skills? 
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Do students who have received special 
services under federal or state programs 
perform better than similar children who 
have not had access to those programs? 

Do students leave formal education with 
positive attitudes toward productive work? 

Human Resource Issues 

The federal government is concerned with the 
flow of human resources to , ssure a work force 
competent to function in an advanced 
technology society and the necessary military 
personnel to protect American interests. 
Planning for human resource deployment is a 
complex process that requires reliable 
information on young people's competencies, 
training, and attitudes.... 

In the past we have vacillated between feast 
and famine in critical personnel areas... 
NAEP should assist governmental and 
educational policy planners by contributing 
information on Jne following kinds of 
questions : 

What are the career goals of high school 
students? 

What are the attitudes of today's youth 
toward the military? toward business? 

To what degree do students with access to 
science and high technology curricula choose 
careers in science more than those with no 
such experiences? 

Are we preparing youth to meet the human 
r ^source needs in the health sciences? the 
humanities? teaching? 

Are vocational/occupational programs 
equipping students with the skills they need 
to function in the work place? 

School Effectiveness 

School administrators are faced with rising 
costs and multiple demands on limited 
resources. They must choose among a host of 
competing interests. Achievement data, to be 
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most useful, should be tied to other 
information to guide policymakers in deciding 
how they might best organize their programs 
and disperse their funds. Although 
achievement is influenced by many factors — 
some school related, others beyond the 
school's control — test data are one measure 
of the effectiveness of schools. Holding 
other variables constant, what factors within 
the purview of school administrators appear 
most likely to contribute to increased 
achievement? How c^n NAEP assist state and 
local policymakers to improve schooling? 

If NAEP is conceived not merely as a social 
indicator, but as a tool to identify problems 
and suggest areas of potentially productive 
research concerning educational progress . NAEP 
should attempt to provide data that address 
the following kinds of policy issues: 

How do pupil/teacher ratios appear to relate 
to achievement? 

Do students with preschool and/or 
kindergarten experiences seem to perform 
better than those without such programs? 

How do particular curricular approaches 
relate to student achievement in reading? 
writing? math? 

What are the relationships of in-service 
training programs, teacher turnover rates, 
and teacher competency requirements to 
student performance? 



The NAEP-related list of questions is not sufficiently elaborated 
to serve the broad requirements of the NCES redesign. As an obvious 
example, it omits questions about the relative effectiveness of 
education in this country vis-a-vis others. A taxonomy of questions 
should include questions about how education is faring not only as 
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compared with education in previous years but also as compared with 

— * vwm^uixowus ua stuucui. d c c urn p 1 1 b rime n c in cne 

United States with that elsewhere, accompanied by information about 
differences in educational practices associated with different results, 
can not only tell us how wel we are "competing," it can also help us 
raise our sights in areas where others may be doing better and lead us 
to examine educational practices elsewhere that seem to be related to 
achieving better or worse results. 

2. General Design Issues 

A clear mapping of the questions to be answered from the data must 
be followed by a decision as to a strategy for gathering the 
statistics. The NCES activity is obviously not conducted in a vacuum. 
Many data-collection programs of other agencies, public and private, 
gather information that is directly or indirectly relevant to 
educational issues. In the single area of "educational outcomes," even 
a partial listing of large data bases that contain information derived 
from tests given in the U.S. contains over two dozen entries; 



Precollege Modai 

Age 



National Assessment of Educational Progress (NAEP) 9 

National Assessment of Educational Progress (NAEP) 13 

Secondary School Admission Test 14 

Metropolitan Achievement Tests 16 
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Iowa Terts of Educational Development 16 
NCES 1980 High School & Beyond (HS&B) Sophomore Cohort 16 

College Board Preliminary Scholastic Aptitude Test 17 

NAEP 17 

Armed Services Vocational Aptitude Battery 17 

American College Testing Service 18 

NCES 1980 Senior Cohort-Base Year Survey 18 

Transcripts of High School Grade (from HS&3) 18 

CEEB Admissions Testing Program (Scholastic Aptitude 
Test, Achievement Tests, Advanced Placement 

Examinations) 18 

High School Equivalency Test 19 

College and Beyond 

Cooperative Institutional Research Program 19 

NCES 1980 HS&B Senior Cohort— First Followup 20 

NCES 1980 HS&B Sophomore Cohort — Second Followuo 20 

NCES Higher Education General Information Survey 

(HEGIS) 20 

Graduate Record Examinations 22 

National Teacher Examinations 92 

Undergraduate Academic Transcripts (from HS&B) (17 to 22) 

NCES 1980 HS&B— Senior Secor dlowup 22 

NCES Recent College Graduate Survey 22 

NRC Survey of Doctorate Recipients 26 
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Other surveys that are pertinent include those that are specialized by 
subject area (e.g. the RTI National Science Survey for ages 6-12), those 
that are international in scope but include the U.S. (e.g. the 
International Surveys of IAEA), and those that provide data only about 
other countries (e.g. the equivalent of our NAEP program, conducted in 
Great Britain). Still others that have developed large-scale longitudinal 
data bases over extended time periods, were sponsored by the U.S. 
Department of Labor in order to track educational and labor force 
activity. (These are found in the DOL National Longitudinal Surveys (NLS) 
of Labor Market experience and the Continuous Longitudinal Manpower Survey 
(CLMS).) Just developing (and maintaining) a good catalog of sources 
would help. 

Trying to nerge these data sets is a daunting challenge but an 
activity that should be studied and tried at least experimentally. A 
system of planned "linking sections" common to different data bases 
might prove feasible and helpful. Even if a complete mei fe ing is not 
feasible, some useful dimensions of comparability using subsets of data 
may be open to discovery and use. At least it should be possible in 
the course of such a study to develop good documentation, available 
centrally, about the comparability of the several tiles, including 
mundane but essential facts such as whether or not the data can all be 
run on the same computer! Such documentation would in itself be 
extremely useful. 

The list includes examples of data from both governmental and 
non-governmental agencies. NCES already arranges to receive most of 
the pertinent data from the government agencies. The statistical 
series produced by non~governm.;ntal organizations in some cases are 
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carefully maintained and are capable of illuminating special areas 
within the particular sphere of interest of the private organization. 
Cases in point are the data sets about college or graduate applicants, 
their test scores and their educational histories that are collected 
routinely by agencies like the American College Testing Program, the 
College Board and Educational Testing Service. The educational 
significance of these data in the public mind became dramatically 
evident in the 1970's when the persistent decline in the mean scores of 
SAT takers was first noted. NCES already draws on some of these 
sources of test -^ta for their information on outcomes. A further step 
might be advance joint planning of issues that could be explored more 
effectively through cooperative arrangements similar to the agreements 
with SEA's or to existing interagency agreements 
within the Federal Government. 

The thicket of problems becomes even thornier when one goes beyond 
data sets in a single broad domain — educational outcomes — and 
includes the many areas touched on in current population surveys by the 
Census Bureau and workforce surveys by the Department of Labor. Since 
the information in these data sets was not gathered on similar samples 
by asking a consistent set of questions, the of NCES in trying to 
bring it together in relation to education issues is extremely 
difficult. The ideal (from one standpoint) of achieving complete 
comparability across data sets is impractical. The basic need to 
maintain continuity of long-running data sets is by itself a major 
deterrent to precipitate change. Nonetheless, efforts should obviously 
be made to remove unnecessary barriers to our ability to pool data 
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across agencies. The present juncture, when NCES is in the redesign 
process, would seem to be a good time to explore the presently 
attainable degree of planned compatibility of efforts without 
jeopardizing the unique needs of each participating agency. 

Frequently one finds that two surveys include questions intended 
to represent a whole complex of information describing a construct of 
common interest, such as socio-economic status, but have selected 
different questions. It would be worthwhile for an interagency 
authority such as OMB to study the extent to which the answers to 
different questions can be taken as valid surrogates for the broader 
construct . 

Notwithstanding the serious obstacles to attaining compatibility 
across data files, we recommend that NCES take th e lea c* in exploring 
with other agencies, public and private, the feasibility of achieving 
greater compatibility among data sets . An effort should be made to 
increase compatibility in the short term where possible or over a 
longer period where that is required. It may be that some highly 
desirable steps toward eliminating redundancy of effort that cannot be 
completed in the near future could be accomplished in 5, 10 or 15 
years if started now. 

3. "What" Studies and "Why" Studies 

No matter how successfully NCES may be able to draw upon extant 
data bases to meet its needs, it will still need to make a very 
extensive primary collection effort of its own for several reasons: to 
fill in the gaps in some areas, to acquire data in a form compatible 
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with its other data elements, or to fulfill its role ?s the principal 
data source in educational areas central to the purr ;>ses of NCES. 

Many of the statistics collected (e.g. per pupil expenditure in 
public schools by state) are facts that have face validity as importait 
in their own right. They answer legitimate questions of "what is." 

Tht stati tics on "what is" are more useful in answering questions 
about present coaditions than in suggesting how or why those conditions 
came about. All too often, people juxtapose two or more sets of dats 
about disparate conditions, find some instances of apparent 
correspondence, and infer a causal connection. The cross-sectional 
data of annual suiveys are, of course, poor bases for causal inference. 
Much better for answering "why" questions are the kin'' of data 
gathered in periodic studies such as the National Longitudinal Study of 
High School Seniors ,f 1972 (NLS) and the High School and Beyond Study 
of 1980 (HS&B). 

The large scale longitudinal studies are proving to be critical in 
lluminating issues of public policy, since they provide a basis for 
tracing the later correlates of earlier student experiences, and they 
do so in the context of a wealth of background information. The 
background data — about financial support, interests, subjects 
studied, extra-curricular activities, c.nd so on — help in interpreting 
the meaning of changes in attitude or in student learnin^ or <n 
d isions to continue or to drop out, or in changing job aspiration? 
K.' in general and differentially by such variables as sex or race. 
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We endorse strongly the view that the longitudinal studies are a 
uniquel y valuable educational resource and urge that they be designed 
as a long-term anu recurrent element in the NCES data-gathering system . 
Insofar as possible, regular long-term funding expectations (or, 
ideally, commitments) should be established and broadly announced so 
that other agencies could reaonably anticipate answering their 
questions on the basis of a continuing data series rather than feeling 
compelled to establish duplicative efforts. 

Both the NLS and HS&B studies trace the progress of students from 
the high school years forward. In order to increase our understanding 
of what is happening in the pre-college years and why it is happenirg, 
we need a companion study that begins in the pre-school years and 
follows pupil progress through the grades, eventually linking up with 
the HS&B sample in secondary school. We urge that a "Preschool and 
Beyond" longitudinal study be instituted by NCES as early as possible . 
Such a study could be instituted as a stand-alone effort or possibly 
created as a longitudinal sub-study within NAEP. Ideally it should be 
undertaken on a broad national scale but if that approach is too 
expensive, thought could be given to the possibility of mounting it in 
a sample of cooperating states. 

Data and Information 

A legitimate question is how far NCES should go beyond gathering and 
reporting raw data by providing the analysis and interpretation that 
turn data into information. 
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In this country generally, the dramatic improvement of information 
processing technologies is likely to lead to a sharp increase in the 
volume of data recorded, manipulated and presented. We will have to be 
very deliberate in our procedures to avoid swamping audiences with 
undigested data "because it is there," Tlie quality of synthesis and 
interpretation will have to keep pace if we a^e to realize the benefits 
of collecting the data in the first place. 

It is our impression that at present the bulk of NCES's activity 
is devoted to providing data. It is our further impression that a 
growing component of NCES's work is in the areas of analysis and 
interpretation, through visual presentation and commentary in The 
Condition of Education , through the new publication on Indicators , and 
through a variety of special reports on particular topics. We applaud 
the shift in eropnasis toward interpretation and encourage a 
c ontinuation in the same direction . 

We believe ?lso that a strong effort should be made to encourage 
recognition of authorship of NCES's interpretive commentaries — 
another trend characteristic of recent years and one that could be 
carried still further. Signed analyses carry with them appropriate 
professional recognition for staff, with concomitant benefits in morale 
and career advancement. They also create at least some small 
theoretical distance between the responsibility attributable to the 
author versus the agency, even though in times of crisis that dista^.e 
is usually very slight except in a purely academic institution. 
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General Issues of Strategy 

Some of the other stracegic issues to which NCES is no doubt giving 
attention in its review include: 

The periodicity of surveys and creation of public awareness 
of the schedule for collection of annual, biennial, 
decennial, etc. data 

Allocation of resources among cross-sectional and 
longitudinal studies 

Proportion of budget allocated to domestic and international 
surveys 

Reliance on NCES's own efforts versus dependency on data 
collected by others 

Extent to which data drawn from other agencies will be based 
on cooperative pre-planning versus serendipitous discovery 

Desirability of creating an interagency mechanism for 
coordinating data collection plans 

The matter of how t o encourage widespread use of the NCES data deserves 
intensive review. Possibilities such as more extensive use of networking 
to make the data readily available need continuous review as the 
available technology advances. Obstacles to, an. techniques for 
encouraging, public use of data tapes should be explored, as should 
mechanisms for sharing insight and problems; e.g. creation of an AERA 
Special Interest Group for people using the High School and Beyond data 
sets, or creation of a consortium of data base users. 



4. Spe cific Design Issues 

The foregoing comments have been concerned with general issues of design. 
A more specific set of issues is posed if one asks a question like 
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"what's wrong with the data we collect now? This is essentially the 
approach taken by Cooke, Ginsburg and Smith in their useful paper,* 

Anyone who has worked with the NCES data or virtually any other data 
set will recognize and be able to add to the deficiencies reported in 
"The Sorry St^te of Education Statistics The best way to improve the 
situation is a question of another order. 

Many of the difficulties noted by Cooke, Ginsburg and Smith stem 
from NCES's dependence on the 50 states to collect and report data 
consistently. The cooperative arrangement with the SEA's has obvious 
advantages. Some agreement on common definitions seems essential, 
however, if the aggregated data are to be meaningful. It is suggested 
that NCES work through the Council of Chief State School Officers to 
procure comparable data from the SEA's . This need not disturb the 
individual state's internal definitions of variables such as attendance. 
A viable procedure might be to arrange for access to all of the raw data 
(e.g. numbei of enrolled children, number absent with excuses and without 
excuses, etc.) from which the SEA and NCES (or CCSSO for NCES) coulc 
derive statewide statistics to fit their own definitions. It would seem 
appropriate for NCES to stand ready to provide technical ass istance to 
states that request consultation on the best ways of collecting a nd 
presenting their data. 



Cooke, C. , Ginsburg, A., & Smith, M. "The Sorry State of Education 
Statistics," January 1985. 
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Some problems, like students* tendencies to overstate their course 
load, probably cannot be eliminated. It is suggested, however, th?t a 
continuing series of studies be undertaken of chara cteristic student 
response bias in key areas as a basis for deriving r esponse adjustment 
coefficients . These coefficients could be applied systematically to 
provide more valid estimates of the trpe situation. The studies needed 
to obtain estimates of response bias would be intensive small-scale 
studies that would need to be repeated perhaps every 5 or 10 years, 
depending on the index. In some cases where the discrepancy between the 
response and the factual situation seeded extreme (e.g. where 80 percent 
of high school seniors report that they have taken a geometry course 
compared with 25 percent shown on transcripts) the most useful result 
would be clues as to how to revise the question rather than calculation 
of a response adjustment coefficient. 

***** 

ETS is well aware that many of the suggestions made above may 
already be well represented in the procedures or the plans of NCES. We 
decided that in this paper we should err on the side of inclusiveness at 
the risk of redundancy. We will be glad to clarify points that need 
further explanrtion or to elaborate on iJeas that may need exploration in 
detail. 
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June 11. 1985 



Mr. Emerson J. Elliott 
Administrator 

United States Department of Education 
National Center for Education Statistics 
1200 19th St. NW 
Washington, DC 20208 

Dear Mr. Elliott: 

The elementary and secondary schools sponsored by congregations of The 
Lutheran Church — Missouri Synod are interested in cooperating In the re-design 
of the education data program provided by the national center for education 
statistics. At this time we are not providing a formal paper for 
consideration in the first synthesis, but we wanted to be sure you were aware 
that we wish to join in the re-design and in the program itself. 

It is helpful for us to know approximately how many non-public schools exist 
at what levels (preschool, eleme^cary or secondary), how many children they 
serve, and how many teachers and administrators serve them. It is also 
helpful for us if we can separate The Lutheran Church — Missouri Synod schools 
from the other schools in your non-public school survey, and that we can 
compare their responses with those of the other church and non-church oriented 
private schools. It wouJ i also be helpful if we could compare data with the 
public schools. 

Although we collect and disseminate rather extensive data on the schools of 
The Lutheran Church — Missouri Synod, receiving specific reports from over 95X 
of our schools, we are very interested in cooperating in this venture. We are 
eager to provide data for important surveys, such as the private school 
survey. In return we appreciate receiving the results of that survey so that 
we can "ontinue to improve our schools. 

It is important that the number of teachers in our schools be counted in the 
survey of teacher demand and shortage. Frequently the non-public school 
teacher demand and shortage is quite different than that found in the public 
schools. Information comparing both types of school would be appreciated. 

One of the growing agencies in the schools sponsored by our church is extended 
daycare. This may become a service offered by public schools in the near 
future if federal financing should become available. I believe that 
information covering those schools which provide extended daycare before and 
after school would be important to be added to your statistics. 
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Mr. Emerson J. Elliott 
June 11, 1985 
page 2 

11 I can help any further or if you have any questions about our data or data 
collection, please feel free to contact me. 

Serving the Master Teacher, 



Carl J. Moser, Appelate Secretary 
Elementary and tiary Schools 




c.c. Dr. Vic Constien 
Dr. James Boldt 
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INTRODUCTION 



The Carl D. Perkins Vocational Education Act (P.l. 98-524) continues the 
information systems establisi d by the 1976 amendments to :he Vocational Educa- 
tion Act of 1963 (P.L. 94-482): a vocational education data astern and the 
National Occupational Information ( ,ordinating Committee. By these continu- 
ations congress has reiterated its long-standing interest in better information 
both to assess the effects of the federal role in vocational education and to 
improve the working cf the labor market. Any information systems established 
for vocational education ,nust attempt to respond tu these two objectives, but 
it is doubtful if any one system can do both. 

The initial attempt to fashion such a system by tne National Center for 
Education Statistics (NCES) failed because it tried to do too much. The origi- 
nal Vocational Education Data System (VEDS) attempted to provide information 
both for policy and for labor market purposes at a level of detail th^t local 
and state sources could not supply with acceptable accuracy. The internal and 
year-to-year inconsistencies in the data assembled by VEDS lea tne Office of 
Management and Budget to stop the collection of 1983-84 data. 

NCES is currently trying to design a new "national vocational education 
data reoorting and accounting system" which will comply wUh the mandates in 
the Perkins Act (Sec 421). These mandates are much the same as those in the 
1976 amendments except there is a greater emphasis on special populations and 
on the use of sampling to collect data. This paper is intended to assist NCES 
in this process. It reflects the ideas of those staff members of the National 
Center for Research in Vocational Educationl who have worked most closely 
with the available national data on vocational education. First the problems 
that the original W EDS encounte;ed are reviewed. The paper then presents 
recommendations for improving th? operation of future systems and the utility 
of the data they collect. 

Problems with the Old VEDS 

A 1979 report by the National Center reviewed the early implementation of 
VEDS and concluded: 
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VEDS future is still uncertain, 

, , . to the degree VEDS is implemented, vocational educators and 
decis ion-makers at every level will have a knowledge of wno enrolls 
in vocational education programs, what happen3 to them afterward, 
and what it costs in the kind of detail long needed, but never 
before available. The extent to which it is implemented will 
depend upon the decisions of thousands of local and state adminis- 
trators as they attempt to supply information in the form that VECS 
requires (pp. 64-65) 

By 1984 the uncertainty had been removed, VEDS as NCES had originally 
tried to implement it was stopped by 0MB, VEDS had generated data at a level 
of detail not previously available, but the dala were not consistent or 
credible. Comparisons of VEDS data to other sources yielded in a few states 
vocational enrollments that exceeded total secondary enrollment. Year-to-year 
changes within states in program enrollments of 50 to 100 percent were not 
uncommon. It would be easy to attribute the poor quality of these data to 
resistance or ineptiti.^ among the data providers 1 more fundamental problems 
underlie most of these reporting difficulties. 

Definitions 

The primary difficulty at the secondary level is definitional. What 
criteria should be used to define vocational students? The quick answer is 
course enrollment: Students who take vocational courses are vocational 
students. By this definition, however, virtually all secondary students are 
vocational. (Campbell, Orth and Seitz (1981) have shown that over three- 
fourths (78 percent) of students take one or more courses designed to teach 
skills for paid employment. If consumer and homemaking and industrial arts 
courses are included, over 90 percent of students take at least one vocational 
course (Meyer 1981, NCES 1984), 

Another technique frequently used is to ask students to classify them- 
selves as to their main course of study. When comparison are made between 
self-report and other classifications made by administrators (Fetters 1975) or 
from an analysis of transcripts (Campbell, Orth and Seitz 1981) approximately 
one-third disagreement between the sources is found. 

If additional criteria are applied to cojrse taking data, such as total 
number of courses, areas of concentration, and the grade level at which the 
courses were taken, it is possible to distinguish those who appear to be 
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preparing for entry into identifiable occupations from those who appear to be 
taking vocational courses for exploratory or avocational reasons, Th^s is the 
approach that Campbell, Orth and Seitz (1981) followed and it enabled them to 
identify five discrete patterns of participation in voca ,onol courses. These 
patterns distinguished Detween those who seem to be preparing for employment 
and others who took vocational courses without appearing to have employment as 
an objective. 

The Campbell, et al. approach is applicable when students have completed 
or left high school and coirplete information is available on the courses they 
had taken. It is less appropriate for classifying students while they are 
still in high school. A student may, for example, take agricultural courses in 
tne ninth and tenth grade, switch to auto mechanics in his eleventh grade and 
to carpentry in the twelfth grade. Depending on when the student was counted 
in his high school years, he would be classified in three different program 
areas. The Campbe 11 et al. decision rules would classify such a student as a 
concentrator-explorer—one who initially appeared to follow an area of spe- 
cialization but who left it to sample other occupational areas. 

Reporti n g System 

A second major problem the old VEDS encountered was the varied and decen- 
tralized nature of the system that generated the VEDS reports. The VEDS forms 
were distributed to the states. The information th-it was aggregated and re- 
ported on those forms was collected from local educational agencies by a vari- 
ety of mean". A few states virtually duplicated the VEDS forms and required 
th9 local agencies to complete them. Some states relied on individual student 
records which were completed at the local level and aggregated at the state 
level. Most states, however, tried to adopt their existing information systems 
to supply the information required by VEDS. The success of thi s approach 
varied widely across states. 

Relying on such a decentralized system requires ve'-y good communication 
from the federal to the state and from the state to local levels. Even when 
the communication is good, there is an inevitable time lag between the initia- 
tion of a request at the federal level and the response at the local U v e1. 
Any changes in the request, and there were nany in VEDS brief history, com- 
pounds th2 communication difficulties. The repeated message that National 
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Center staff received when they contacted VEDS coordinators in the states was 
to stabilize and simplify the system. Of the two stability is probably more 
important than simplification. 



Special Needs Populations 

Vocational educators have had difficulty in developing accurate ways to 
identify disadvantaged and handicapped students for reporting purposes. These 
difficulties stem primarily from the discrepancy between the way special popu- 
lations are served in schools and the way they are defined in legislation. The 
Perkins Act, for example, does not limit the definition of disadvantaged to 
income, It includes individuals "who have economic or academic disadvantages 
and who require special services and assistance in order to enable them to 
succeed in vocational education programs," [Sec 521(12)]. Even if the defini- 
tion were limited to family income, public schools are reluctant to request 
such information. On those occasions when it has been requested, the schools 
have encountered resistance and protests. Consequently schools have had to 
rely or. proxies of disadvantaged status, such as eligibility for free or 
reduced price school lunches. Even with such proxies, the names of those 
eligible are not widely shared and often are not available to the individuals 
responsible for completing the local forms that are aggregated for VEDS, 

The existence of an individualized educationa 1 plan should be a clear 
indicator of whether a student is handicapped. Vocational administrators often 
claim, however, that many Handicapped students are mainstreamed in vocational 
classes without the teachers or the administrators ever being informed of the 
students 1 handicaps. This is especially true among learning disabled and 
speech impaired students who constitute approximately two-thirds of all handi- 
capped students, T heir handicaps are loss obvious and often less of a detri- 
ment in vocational classes. Furthermore, by the definition contained in the 
Perkins Act [Sec 521(15)], these students must require special education and 
supportive services to succeed in regule- vocational classes to be considered 
handicapped. A literal interpretation of this definition means if they are not 
receiving special services, thpy should not be reported as handicapped. 
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Recomme Nations 
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for a division between data collected primarily for policy purposes and data 
for labor market information. Most data for policy purposes can best be col- 
lected with special studies conducted on a sampling basi s. Accurate data on 
program completers for the Occupational Information System, however, requires a 
census, a complete enumeration of the population of interest. 

Data tor Policy 

Policy questions are basically concerned with who is served, how they are 
served, at what cost, and with what effects? Data to answer these questions 
can best be collected with specially designed questionnaires from representa- 
tive Samples of schools. The two on-going l^igitudinal studies of NCES and the 
one currently being planned can provide much of the needed data. Analyses of 
data of this type can provide far more precise information on the character- 
istics of vocational students, their secondary and postsecondary educational 
experiences and subsequent work careers than any aggregate reports. 

The use of high school and postsecondary transcripts to define varying 
patterns of participation in vocational courses is recommended. Such a prac- 
tice will deal with the definitional problem that plagued the old VEDS, The 
collection of original data from respondents in selected schools overcomes the 
difficulties of using the varied educational reporting systems in the seoarate 
states to generate the data. 

Future longitudinal studies should be supplemented to provide more inf 
mation on the educational process. Indicators of the educational process 
within vocational education could include: 

• Student recruitment, selection 

• Sources of curriculum 

• Use of class time 

• Relevance of equipment to that being used by employers 

• Contact with business and industry 

• Background of instructors, most recent experience in occupational 
areas they teach 

Information for some of these indicators could come from the students as 
well as from teachers and administrators. Cost information can be obtained 
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from the financial reports filed with the Office of Vocational and Adult 
Education, 

Labor Market Information 

Publ ic vocational education is the major source of i n format ion on the 
supply of new workers for the Occupational Information System developed by the 
National Occupational Information Coordinating Committee, To provide this 
information at a level of detail sufficient for state and local planning 
decisions requires a census of all students who complete or leave vocational 
program after attaining a competency level judged suitable for paid employment 
in specified occupations. To attempt to provide these data on a sampling basis 
would require so many primary sampling units that the costs of collecting the 
data would be prohibitive. 

It is recommended that instructors in public vocational programs complete 
a standardized form for each program completer or leaver. These instructors are 
in the best position to make such judgments on the competencies of their stu- 
dents. The form the instructors complete should contain background information 
on the sex, age, race/ethnicity of the students, 2 the program area in which 
trained and whether or not the individual attained a competency level suitable 
for employment in specified occupations. Such a report would deal with the 
problem of defining a program completer or leaver which is especially trouble- 
some at the postsecondary level. Most postsecondary students do not enter a 
specified program. They take one or two selected courses to meet personal 
needs. Many of them are employed at the time they take these courses. 

The forms instructors complete would be submitted at periodic intervals by 
local educational agencies to their state offices. At the state level, the 
forms would be aggregated for use in the state occupational information system 
and a cumulative report made each year to NCES. The National and State Occupa- 
tional Information Coordinating Committees would like program completers to be 
reported al the six digit level according to the Classification of Instruction 
Program (Malitz 1981) code. This level does not reflect the way most voca- 
tional programs at the secondary and pcstsecondary level are offered. These 
programs are designed to provide preparation for employment in a number of 
related occupations. To require reporting at the six-digit level forces 
individuals to make arbitrary cho ; :es that cause unreliability in the data. 
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The six-digit level may be appropriate for some short-term retraining or 
specific. 

The total number of programs completers reported by public vocational 
programs should not be entered directly as data on the supply of skilled 
workers. Campbell, Gardner and Winterstein (1984) have found that less than 
half of secondary students who complete extended vocational programs actually 
seek employment immediately after high school. Many go on for additional post- 
secondary education. The results of this and future research should be used to 
adjust the supply data for the Occupational Information System so that the 
completer figures more accurately reflect those who actually seek employment. 



FOOTNOTES 

1. In subsequent discussion National Center shall refer to the vocational 
center and NCES to the statistical center. 

2. These data would be for policy not labor market purposes. 
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Suggestions For The NCES Redesign Project 

The National Education Association (NEA) is entering the information 
age with a lack of information. In a nation accustomed to numbers, this 
statement seems absurd. After all, public and private collections of 
educational data exist. Statistics bombard the education market weekly. 
The public press turns out bales of educational reports yearly. 

All of this information should provide a cl2ar picture of the status 
of education in this country. It should provide sharp refleccions of the 
way education is changing. It should also help answer with increasing 
sophistication the many questions of cost, benefit, and quality. Avail- 
able information, however, does not reflect well the educational 
landscape. 

Bringing education into sharp focus is as difficult today as it was 
twenty years ago. The need for a sharp focus, however, is perhaps greater 
today than it was in the past. Government regulation of education has 
increased. State furding for education has increased. The number of 
students enrolled in schools is increasing. And new measures to reform 
education are everywhere present. 

All of these changes carry with them a demand for more and better 
information about education. Yet the current supply of data has not 
kept up with the demand, and educators have reason to worry. If the 
demand is not met, then future policy will likely be based on a murky 
picture. Furthermore, we will not know with any certainty what impact 
effort3 to reform education have had. 

The redesign project of the National Center for Education 
Statistics (NCES) offers a partial but significant solution to the 
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information problem. The Center's resources, capabilities, and pro- 
fessional reputation are ideally suited for the data-collecLiun efforts 
needed in education. Although the NEA does not believe NCES can or 
should solve ail the information problems, we do believe it can solve 
some of them. For this reason, the NCES redesign project is of major 
importance, and the NEA is pleased to be involved. 

NEA suggestions for the NCES redesign of its elementary and 
secondary education data program are organized below under five cate- 
gories suggested in NCES guidelines. The categories are: Issues and 
Data Needs; Data Modifications; Importance of NCES Series; Data Deletion; 
and Relevance, Quality, and Utility. 

Issues and Data Needs 

Today, our national self -percept ions of education are regularly 
confirmed or challenged by statistics ou many matters. Whether the 
meanings read into the data are reasonable or fanciful, the numbers 
provide a basis for popular and specialized discussion. 

The NEA expects official numbers, especially those that appear in 
series, to play an increasingly prominent role in policy deliberations. 
Among the many issues likely to be discussed in the future, several 
noted below seem particularly amenable to NCES collection efforts. 

Effective Schools 

Issues of quality education will likely expand to include recent 
effective schools research. Subjects of interest can be expected to 
include characteristics of school organization, school governance, 
school administration, public expectations for schools, federal and 
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state regulations, and local policies. 

The current NCES data program will noi provide sufficient support for 

the effective schools issue. The following revisions should help 

strengthen the data base: 

o Add to both the public and private school surveys 
data elements pertaining to the school character- 
istics of organization, governance, administration, 
expectations, regulations, and policies. Suggestions 
for such elements appear in Tables 1 and 2 at the 
conclusion of this paper, 

o Consider expanding the NCES program to include 
case studies, field studies, policy reviews, 
historical research, and additional surveys to 
expand the scope and detail of effective schools 
data. 

Equity 

Since World War II, numerous policy changes concern the issue of 
equity. School busing (to adjust the numbers of white and black students) 
and job quotas (to ensure the efficacy of affirmative action) are but two 
examples. 

Equity issues pertaining to race and sex will likely persist. Issues 

pertaining to age and ethnic origin will likely grow. The following 

suggestions anticipate the demand for more detailed data where they are 

not already gathered: 

o Refine the varxants of Spanish ethnicity to 
include Mexican American, Puerto Rican, and 
Cuban ior all NCES surveys. 

o Refine the variants of Asian ethnicity to 

include Pacific Islanders, Japanese, Chinese, 
and Vietnamese for all NCES surveys. 
Consider using the U.S. Census item for 
this refinement. 

o Add the ethnic category of American Indian 
and Alaskan Native for all NCES surveys. 
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Public and Private Schools 

A broadened perspective on schooling is eviden* in current policy 
discussions. This perspective includes K-12 schools in the mixed 
public-private system of schooling, pre-primary schools, and adult 
education and training programs in both public and private sectors. 

This perspective raises many questions about public and private 
schools: how they are sinilar, how they are different, and what they 
can learn from each other. The following redesign suggestions antici- 
pate a continued and broadened public-private school debate: 

o Add -o the Sample Sur/eys component a Pre-Pri:nary 
School Survey designed to gather information 
about the location, organization, program, 
governance, finance, employees, and students of 
these schools. 

o Align the data elements in the Private School 
Survey and Public School Survey so that the two 
surveys are comparable. Specific suggestions 
for revisions appear in Tables 1 and 2. 

o Expand the scope of data collected for Doth the 
private and public school surveys. Suggestions 
for expansion appear in Table 2 concluding this 
paper. 

o Add to the Sample Survey component a survey or 
to the Other Agency Data component survey items 
that track the magnitude and growth of adult 
education and training programs in both public 
and private sectors. 

School Finance 

Limited resources require that money be spent on education wisely 
at the local, state, and federal levels. Current revenue and expenditure 
data seriously curtail the kinds of questions that can be asked and 
answers that can be exploied. Because the demand for cost-effectiveness 
studies will likely increase in a period of limited resources and fiscal 
restraint, the need for mere detailed revenue and expenditure data will 
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grow. The following suggestions for NC^ZS revision can ease the need 



for bercer fiscal data: 

o Provide greater revenue and expenditure detail 
for both private and public schools. See Tables 
1 and 2 for specific suggestions. 

o Add data for incentive plans and salaries of 
public and private school administrators and 
educational support personnel. 

Student Outcomes 

Statistics are regularly published on such fundamental matters as 

reading and literacy rates, achievement rates, and dropout rates. The 

inadequacy of measures for each of these rates is well established and 

well known. The following suggestions call for a major revision of 

student outcome data: 

o Expand and standardize the definition of student 
performance outcomes to include more knowledge 
areas. Consider such categories as linguistic, 
musical, logical-mathematical, spatial, bodily- 
kinesthetic, and personal knowledge. 

o Develop measures appropriate for an enlarged view 
of student outcomes. Consider the possibility of 
building upon the diversified measures developed 
for the National Assessment of Educational 
Progress. 

o Collect fall and spring enrollment figures. 

o Standardize definitions of dropout, attendance, 
and literacy. 

o Convene an advisory group to study methods suit- 
able for measuring dropout rates and stuuent 
mobility. 

Teaching Quality 

For a number of years, fairly simple models directed the collection 
of dna pertaining to teaching quality. For example, some models were 
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constructed on the metaphor of an assembly line with students and proce- 
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which teachers had absolute control. Other models, including those 
underlying several reform proposals, take an economic view of teachers 
and teaching. 

Accumulated research and the experiences of teachers indicate that 

all of these models misrepresent reality. For this reason, the issue of 

teaching quality will likely become more complex as models for thinking 

about quality change. Suggestions consistent with this change include: 

o Add data elements for each of the following known 
components of effective teaching: personal char- 
acteristics of teachers, teacher competence, teacher 
performance, student learning experience, student 
learning outcomes, teacher credentials, school 
context, characteristics of students as a class, 
and characteristics of students as individual 
learners. Suggestions for data elements appear 
in Tables 1 and 2. 

o Consider convening an advisory panel to develop 
and refine over time measures of teaching 
effectiveness. 

o Consider expanding the NCES program to include 
case studies, field studies, policy reviews, 
historical research, and additional surveys to 
expand the scope and detail of data for the 
components noted above. 

Data Modifications 

NCES data bases should be expanded to provide a broader and more 
detailed source of information about K-12 education. Specific recommendations 
for expansion appear in Tables 1 and 2. Table 1 identifies data elements 
that need to be added to existing data collection components. Table 2 
identifies data elements that should be collected through additional 
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surveys. Additional surveys are needed to provide a more comprehensive 
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conditions, and outcon) ^ of K-12 education at the local level. Suggested 

data elements in the table would be useful in providing estimates of: 

o A wider array of local conditions related to school, 
teacher, and pucll performance. 

o Variation within schools, districts, regions, and 
the nation in the presence of conditions likely to 
influence education quality. 

o Many dimensions of education practice at a given 
time and place. 

A major desirable outcome of expanding the NCES program will be an 
improved data base for professional, governmental, and public decisions 
regarding ways to improve the quality of K-12 education. Such data can 
contribute to a better understanding of the complexity, variation, and 
similarities of K-12 education throughout the nation. The data can be 
used to study problems, opportunities, and decision options related to 
improving education. The data can also be used to study departures from 
traditional approaches to funding, accrediting, evaluating, and changing 
public schools. 

The utility of NCES data bases can also be increased if the 

following criteria are consistently met: 

o Conditions in education are accurately measured and 
reported. 

o Data are collected with instruments that meet high 
technical standards . 

o Data are collected under normal conditions. 

o Data are made available to users in e timely fashion. 
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Importance of NCES Series 
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decisions by government and nongovernment users alike. The need for 
these data has escalated during the eighties as researc' 2rs, legislators, 
and the public focus on issues of educational improvement. 

The importance of NCES T statistical programs ha g^own in conjunction 
with this expanding need. Consumers of education data have come to view 
NCES data series — all series — as accurate, non-biased sources of informa- 
tion with which to address an increasingly complex education enterprise. 
Whixe each NCES data series has a wide audience of users and may be con- 
sidered as essential to the planning and design of public education 
policies, the Common Core of Data may represent the most heavily used 
series of public school statistics. 

The Core is the cornerstone of educational information in the United 
States. No other public or private institution collects and maintains 
public education data to the extent that NCES does via the Core. Several 
groups — including the NEA — conduct data collection activities which 
parallel the Core in some respects, but these efforts pale in comparison 
to the NCES program. Fcr this reason and others, the support and main- 
tenance of the Core component should be a national priority. 

The Core represents the most basic data series within the NCES. 
It enables assessments of what was, what is, and what will be in a 
statistical sense. Annual updates to Core surveys provide basic statis- 
tical information on public schools, their pupils, personnel, and finances. 
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This information, by school, LEA, or state, serves as an invaluable tool 
for measuring change, signaling trends, and designing future education 
policies. Data from the Core are used as benchmarks, goals or objectives, 
and measures of success and failure. Simply stated, the Core allows us to 
"know things 11 about public education, and thereby allows us to "do things" 
about public education. 

The Core stands ready to support basic research, budgeting decisions, 
and programmatic planning. It can unccver questions and issues requiring 
further investigation. Data collected through the Core, then, are of 
Immense value in and of themselves, but they also serve as means *o the 
achievement of numerous goals. 

Educational issues of concern to government officials, educators, and 
the public throughout the remainder of the twentieth century will call for 
education information as provided "hrough the Core. Indeed, selected 
variables from the Core can and have been used to help assess the outcomes 
of recent reform initiatives in various states. Thi? evaluative process 
is vital to the prospects for meaningful reform of public education offerings. 

Data Deletion 

The NEA recommends against any reductions or elimination of existing 
NCES data series. We believe that all the series and items are vital to 
the interests of the educational community. 

Relevance, Quality, and Utility 

The summary documents and data tapes provided by the NCES have 
been useful to the NEA. NCES 1 access to school and school district data 
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and their data collection processes cannot be matched by any nongovernmental 
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in service to the e meat ion community. NEA does have specific recommenda- 
tions for improving the data, the collection process, and the dissemination 
of the data in order to increase the relevance, technical quality, and 
utility of the data programs. 

The relevance can be improved b y providing more timely data, access 
to more raw data, consistency between public and private school surveys, 
and consistency in surveys over years. The speed wit 1 ! which the data are 
made available to the public is critical. The availability of the raw 
data, on tape, permits NEA and others to perform their own analyses. The 
use of the same questions for public and private school surveys permits 
more extensive matching and comparing of school systems on a wide variety 
of attributes. T same issue of consistency applies to surveys repeated 
over the years. The use of the same questions permits an analysis of 
trends. 

The technical exility can be improved by ensuring complete and 
accurate documentation, more complete editing of the data, and increased 
efforts to eliminate missing data. Data provided on tape should be 
thoroughly reviewed for errors in record descriptions and data documenta- 
tion. More comprehensive edit checking would reveal inconsistencies in 
the data. For example, the computation of ratios between certain items 
such as enrollment and teachers would highlight unreasonable data con- 
figurations that do not appear in individual items. The use of random 
audits for individual schools and d-^tricts may reveal ambiguous data. 
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or areas of difficult data gathering. For example, a review of individual 
districts may reveal that the definitions of staff categories used by 
districts do not match those of NCES, Efforts to revise or to promote 
use of those codes could then be undertaken. The elimination of missing 
data could provide more accurate summary data. 

The usefulness of the data can be improved by providing more docu- 
mentation on the availability of NCES data and more contact with NCES 
personnel for future survey planning, NEA needs to know ^hat data are 
available from NCES, in what forms the data are availa.^, and when the 
data are released. Increased contact between NCES and the user community 
will enhance the use of present data and the planning of future surveys. 
The process for future survey planning that is now being implemented is 
?n excellent step and should be maintained. 
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Table 1. Summary of NEA Suggestions for Additions and Changes for 
NCES Data, By Census and Survev 



NCE S Component 



Common Core of Data 



N EA Suggestions 



.. Public School Universe 



o Add spring membership. 

o Add full-time-equivalent classroom teacher 
by sex and elementary/secondary level. 



2. Local Education Agency 
Universe 



o No additions or changes, 



3. Local Education Agency 
iJonfiscal Report 



o Add fall membership by grade. 

o Add number of full-time-equivalent LEA 
employees in all employee categories. 

o Add number of full-time-equivalent teachers 
by individual grade. 

o Add presence or absence of collective 
bargaining agreements for teacher, 
administrator, and educational support 
personnel groups. 



4. Public School District 
Finance Report 



o Provide revenue by source consistent with 
NCES handbook on financial accounting. 

o Provide expenditure by function consistent 
with NCES handbook on financial accounting. 

o Provide other uses of funds by category 
consistent with NCES handbook on financial 
accounting . 

o Prov' ie special exhibits by category 

consistent with NCES handbook on financial 
accounting , 



5. State Aggregate Nonfiscal 
Report 



o Add fall membership by individual grade. 
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Table 1 continued 



NCES Component 



NEA Suggestions 



tsonrxscaj continued 



6. State Aggregate Fiscal 
Report 
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by major assignment category, by state, 
o Add number of high school graduates. 



o Make revenue, expenditure, other uses, 
and special exhibits detail consistent 
with revisions suggested for district 
finance data. 

o Add average daily attendance. 

o Add state law defining average daily 
attendance. 

o Add state aid formulae. 



Sample Surveys 

1. Private School Survey 



o Add fall membership by individual grade. 

o Add total membership. 

o Add design capacity of school. 

o Add ethnicity enrollment as percent of 
total enrollment. 

o Add grade span. 

o Add teacher college credits by subject 
matter field. 

o Add information on additional training 
for teachers. 

o Add personal characteristics of teachers. 

o Add membership by major subject. 

o Add average SAT/ACT scores and percent 
tested by school. 

o Add teacher incentive plans. 

o Add teaching assignment and classroom 
enrollment for teachers . 
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Table 1 continued 



NCES Component 



NEA Suggestions 



Private School continued 



2. Public School Survey 



3, Recent College Graduates 
Survey 



4. Survey of Teacher Demand 
and Shortage 



o Add teacher hours per week by activity, 

o Add number of volunteers by activity 
category, 

o Add fall membership by individual grade, 

o Add highest degree earned by teacher, 

o Add number of years experience by teacher. 

o Add admission requirements, disciplinary 
policies, length of day and school year, 
and other characteristics consistent with 
private school survey, 

o Develop better descriptors for this data 
set, 

o Add SAT/ACT scores. 

o Add academic program/preparation detail, 
o Add more demographic detail, 

o Provide rationale for number of private 
schools in survey sample, 

o Add number of teachers leaving and why. 

o Add number of full-time-equivalent teachers 
by grade, 

o Add class size by type of class, 

o Add number of budgeted positions. 

o Add number of needed positions, 

o Add descriptions of recruitment and 
employment practices. 

o Provide rationale and greater detail for 
teacher incentive plans. 
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Table 1 continued 



NCES Component 



N EA Suggestions 



:>. tiign ocnuui Aim 
Beyond 



6. Library/Media Center 
Survey 



o Add descriptions of teacher and student 
recruitment and placement policies. 

o Add more male guardian questions on the 
parent questionnaire — whether he was 
present during pre-school years, worked, 
etc. 

o Add community descriptive elements such 
as racial/ethnic mix, community size. 



o Add number of books, materials loaned. 

o Add number of computers, programs available. 



Other Agency Data 

1. Preprimary Enrollments 
of Children 3-5 
Years Old 



o If this survey is repeated, add length of 
program day, program year. 

o Add enrollment and placement requirements 
for students. 

o Add health care services descriptions. 

o Add state law governing provision of pre- 
primary education. 

o Add state law governing attendance. 

o Add more program description. 
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Table 2. Recommended Data Elements By Major Category For New Or Expanded 
NCE5 Sample Survey Component: 



A. Students 

1. Fall membership by grade 

2. Enrollment by subject matter 

3. Students qualified for special programs 

4. Students enrolled in special programs 

5. Average SAT/ACT scores and percent of students tested 

6. Ethnicity status and percent of total enrollment 

7. Student transfers 

8. Student dropouts 

9. Student attendance 

10. Experience with violence 

11. SES distribution 

B. Teachers 

1. FTE by school level 

2. FTE by category (regular, special education, etc.) 

3 . Assignment type (department head , chair) 

4. Tenure status 

5. Job differentiation status (Master teacher, mentor teacher, etc.) 

6. Activities in day 

7. Highest earned degree 

8. Years of experience 

9. Education specialty in college 

10. Salary average per school 

11. Salary intervals per school 

12. Additional training 

13. Credits by subject, continuing education 

14. Marital status 

15. Sex 

16. Age intervals 

17. Race 

C. Administrators 

1. Highest degree earned 

2. Years experience 

3. Education specialty in college 

4. Salary average per school 

5. Marital status 

6. Sex 

7. Age intervals 

8. Ethnicity 

9. Administrator salary sche^'le 

10. Administrator salary by intervals 
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Table 2 continued 



D. Educational Support Personnel 

1. Standard definitions and classifications 

2. FTE by job category 

3. Compensation plan, fringe benefits 

4. Job qualifications by category 

5. Staff development practices 

6. Evaluation practices 

7. Supervisory practices 

8. Demographic data 

E. Finance 

1. Revenue by source (include private and federal grants) 

2. Expenditures by major assignment category: compensation 
and fringe benefits 

3. Expenditures by major classification 

F. Programs and Practices 

1. Length of school day 

2. Length of school year 

3. Programs offered 

4. Class size by class type 

5. Pupil load 

6. Admission policies by type of school 

7. Teacher activities by percent of time 

8. Teacher incentive plans 

9. Teacher edacation programs 

10. Teacher support programs 

11. Teacher evaluation process 

12. Discipline policies 

13. Job differentiation plans 

14. Standardized testing programs 

15. Grading policies 

16. Span of teacher authority 

G. Classroom 

1. Classroom size 

2. Classroom space 

3. Books and materials 

4. Audio-visual equipment, high tech equipment 

5. Available supplies 

H. School Administration 

1. Source of authority (public, private: religious, nonreligious) 

2. Structure (single school, state, private network) 

3. Location of school (city, suburb, rural) 
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NGA COMMENTS ON NCES 
REDESIGN OF THE ELEMENTARY/ SECONDARY 
EDUCATION DATA PROGRAM 



Introduction 



A majority of the governors indicated in their 1984 and 1985 state of the 
state messages that education continues to be a top priority. The general theme 
of the 1984 and 1985 state initiatives has been to improve the quality of the 
education system. The governor as the state's chief executive officer guides 
education policy in primarily three ways. First, the governor performs a 
leadership role in setting the agenda for and promoting activities on 
educational improvements/reform. Second, the governor is responsible for 
developing state budget requests for education expenditures. In many states the 
education budget accounts for as much as 30 percent of the total state budget, 
and third, many governors are responsible for appointing state education board 
members . 

As leaders in setting the state education agenda, governors in 1985 
planned to focus primarily on initiatives to improve the teaching profession, 
address student quality through improvements in math and science instruction, 
and examine and increase the financing of education. This represents a shift to 
more specific strategies from the broader initiatives of the past few years to 
link educational reform and improvements with other state policies directed at 
promoting state revitalization and economic growth. Table 1 indicates that in 
1985 the largest number of states expressed an interest in the issues concerning 
incentives in the teaching profession. Twenty three states emphasized teacher 
career ladders and 21 states emphasized teacher salaries. Of the remaining top 
ten issues, 4 are related to school administration, 3 of these to financing 
mechanisms and 1 related to school administration reform. Another four issues 
are related to student quality including student competency and child abuse. In 
comparison, in 1984 the gov°rnors in 29 states emphasized their interest in the 
broader aspects of building stronger partnerships between education and 
business/industry. Private sector linkages were seen as a means to accomplish 
the goals of preparing a better-educated future work force and thereby 
contributing to the state 1 s future economic growth potential. 
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Table 1 



Top Ten Education Issues Cited in the 1985 
State of the State Messages 

Number of States 



Issue Area Citing 

Teacher Career Ladder and Development 23 

Teacher Salaries 21 

Math/Science Instruction 20 

Finance Formula 20 

Economic Development, Voc. Ed., and Tech. 19 

School Administration Quality and Reform 18 

Funding Increase 18 

Local School Aid 18 

Student Testing and Competency 17 

Child Abuse 17 



Source: Governors' State of the State Messages, 1981 (Forty-seven 
state messages reviewed.) 

During the 1980' s, more than 30 states have enacted major financial 
reforms of elementary/secondary education. In 1984, 25 governors planned to 
enhance education efforts by providing more fiscal resources and 9 governors 
proposed tax increases to provide adequate resources for the improvement of 
their states 1 educational system. The majority of the funding increases 
proposed (20 states) was planned for use in raising teachers' salaries while 9 
states proposed funding for improvements of school facilities and equipment. 
Ten states proposed increasing state aid to local schools systems. In 1985, 20 
governors were interested in finance formula issues, 18 governors plan to 
enhance education efforts by providing more fiscal resources and 18 governors 
emphasized local school aid. This represents a significant shift from the 
1970' s when more than 28 states enacted major financial reforms for elementary 
and secondary education to equalize fiscal resources among school districts in 
order to relieve property taxes and legal challenges. 

State Policy-Makers Data Needs 

In order to perform education policy setting functions, states need to 
plan, develop, implement and evaluate education initiatives. For these purposes 
a combination of state and national data are useful. State px^oduced education 
data which are designed to meet specific state needs provide the core for state 
education policy-making. All state education policymakers rely heavily on 
information from the State Education Department but also on local school 
districts and on education associations. However, national trend data and 
consistent and accurate data from all states for macro comparison purposes is of 
key interest as well. 

The top ten priority issues as identified in the governors' state of the 
state messages can be used to provide a framework for a discussion of data 
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needs. The list of the top ten education topics is provided to give a picture 
of the current education topics of interest to governors. We are not 
suggesting, however, that in trying to determine the needs of the data system 
for elementary/secondary education for the next 10 years, that currently popular 
issues sach as those identified in Table 1 be used exclusively as a barometer of 
long-term data needs. This list, however, does ref back to some broader 
core issues that fluctuate little over the long run. 

The education citations in the state messages can be organized into three 
general issue areas which include improving student, teacher and school 
administrative quality. Table 2 illustrates this organization and ranks the 
topics in each of the three areas. As can be seen from this table, the primary 
topics of interest to improve student quality concern basic education focusing 
on technology and technical skills. Other areas include competency measurement, 
child care and well being, student groups at the extremes (dropouts and gifted) 
and special-need students; and community behavior that affects student outcome. 
The issue of graduation requirements ranks last. The key issue in improving 
school adr^nistration is school finance. Other issues include general 
management initiatives. Of somewhat lesser interest is the issue of teacher 
shortage. 

The list of issues in Table 2 can be examined in terms of more specific 
data items to determine which are of interest to state education policymakers. 
The Education Policy Consortiun developed a preliminary list of potential data 
items of interest to consortium member associations and their constituencies. 
These data items listed in Table 5 are organized for this paper into four 
categories including student data, teacher data, school data and finance data. 
The data items related to the governors' top ten priority issues are indicated 
with an asterisk as are the items generally available through NCES. 

In the first area in Table 3, student data, the governors are not timid to 
talk about measurement of educational outcomes across states. As more states 
move toward preparing a better educated workforce to encourage economic 
development the issue of identifying student outcomes emerges as more than 
assessing student achievement. More data than test scores, such as the SAT, are 
needed to determine post-school experiences. The education process should be 
traced from start to outcome to determine what happens to the in-school 
population upon leaving an education program (by graduating or dropping out), 
what are their post-school labor market experiences in terms of employment, 
unemployment and earnings, and whether they re-enter school at some future 
time. In the future, student outcome measures may be one set of evidence used 
in evaluating education reform policies currently being initiated. 

Longitudinal studies such as the High School and Beyond Survey are one way 
of determining outcome measures. This is one of the few surveys that capture 
data from students on a longitudinal basis. It seems, because of its somewhat 
unique nature that this questionnaire should be a priority to be maintained, 
improved in terms of data quality and potentially be expanded to gather more 
data, in terms of content and sample size to make the data more state specific. 

In the second area, outlined in Table 3, teacher data, the governors are 
currently interested in examining the teaching profession as a primary factor in 
improving the education system. Incentives to keep and attract quality teachers, 
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Table 2 



Summary of Education Initiatives 
Cited in the Governors' 1985 State of the State Messages 

A. Topics Cited to Improve the Quality of K-12 Students 



Number of 

Rank Topic States 

1 Math § Science Instruction 20 

2 Economic Development, Voc. Ed. § Tech. 19 
3.5 Testing and Competency 17 
3.5 Child Abuse 17 

5 Child Care § Early Childhood 15 

6 Dropouts G Discipline 13 

7 Gifted 12 

8 Computer Literacy 11 

9 Community § Parent Involvement 10 

10 Special Ed. § Handicapped 8 

11 Graduation Requirements 4 

B. Topics Cited to Improve the Quality of K-12 Teachers 

Number of 

Rank Topic States 

1 Career Ladder § Development 23 

2 Salaries 21 

3 Certification § Evaluation 11 

C. Topics Cited to Improve Quality of K-12 School Administration 

Number of 

Rank Topic States 

1 Finance Formula 20 

3 Administrative Quality § Reform 18 

3 Funding Increase 18 

3 Local School Aid 18 

5 Management /CI ass Size 13 

6 Facility/Equipment/Text Books 11 

7 Teacher Shortage 4 



Source: Governors 1 State of the State Messages, 1985. (Forty-seven state messages 
were reviewed.) 
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Table 3 

Preliminary List of Education Policy Consortium 
Identified Data Items Related to the To™ Ten Governors 1 Priorities 



Governors 1 NCES 
Top Ten Data 
Topic Data Item Description Priorities Available 

STUDENT Achievement * * 

DATA test scores 

school gr ? es 

promotion cord 

Attainment * 
drop out rates 

post drop out experience 
graduation rates * 
post graduation experience 

In-School Behavior 

attendance/truancy 
vandalism 

suspension/expulsion 

course enrollments * * 

attitudes 

Community Behavior 

voter registration and 
participation 

Individual Characteristics * * 

demographic (age /race /sex) 
SES background 
grade level 

type of school attending 
migrant /refugee 
pr.i ary language 
handicap 

abused as a child * 

TEACHER Training/Certification 
DATA formal education 

participation in: 
in-service training 
pre-service training 
loan/scholarship availability 
certification subjects 

Evaluation of performance 

classroom evaluation 
competency test scores 
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Table 3 (Continued) 



Governors' NCES 

T/-ntn Tnn Flo * o 

1 \jy 1 j^-u v 

Topic Data Item Description Pr iorities Available 



Compensation 

pay for performance 

incentives 

salaries/benefits 



Fmployment Status 

retention rates 

reasons for leaving profession 
working conditions 
years of service 
course assignments 

Personal Characteristics 

demographic (age/race/sex, 
attitudes 

academic talents by employed/ 
leaver 



SCHOOL 
CHARACTER- 
ISTICS DATA Curriculum anu Assessment 

Improvement 

Instructional materials 

Training 

Alignment of curriculum 
within grade, across grade, 
with assessment 
LEA and state testing 
Education indicators other than 
tests 



School Improvement 
strateg ies 
training 

planning process 
curriculum improvement 

State Role 

monitoring/accountability 
technical assistance 
relationship to local cl^s* icts 

Technology 

Community Involvement /Sat isf action with 
Schools 
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Table 3 (Continued) 



Topic 

FINANCE 
DATA 



Governors ' 
Top Ten 
Priorities 



NCES 
Data 
Available 



Source: 



Data Item Description 

State and Local Revenues * * 

tax base level and compo. 
tax rate level and compo. 
total revenues 
sources of revenues 
relief provisions 
revenue limitation 
provisions for non-public 

Expenditures (local) * * 

total expenditure^ 
total elementary/secondary 

expenditures 
expenditures by category 
cost of special programs 
student aid categorical vs. 

formula aid for special 

student population 

expenditure limits 

Federal Aid * * 

total amount by state 
allocation formulas used 

(federal and state) 
state/local split 
administrative services split 
source of match 
$ amount of audit except 
$ amount of carryover 

This preliminary data item list was derived from an Education 
Policy Consortium meeting, February, 1985. 
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such as compensation and career ladders are of key interest. Of particular 
concern is the notion of higher salaries generating more qualified teachers, 
which if validated, would encourage states to change salary structures, increase 
the ffiiriifiiiin and/or average teacher salaries and establish merit or incentive pay 
programs. Of lesser interest to governors is the number of teachers employed 
and teacher shortages. If the profession has attractive incentives, the issue 
of teacher shortage may be of negligable importance. 

The Public and Private School Surveys which both collect information on 
these topics should be among iNCES priorities. The samples should be examined to 
determine the feasibility of expansion to collect data more state specific. 
This should be considered in conjunction with the further examination of 
appropriate state administrative records and a deemphasis on the Teacher Demand 
and Surplus Survey, 

In the fourth area of Table 3, information on school finance has been an 
issue of interest for a number of years and will no doubt continue to be of 
interest as efforts to improve the school system are continued into the next 
decade. While the issue is not a new one, a shifting focus overtime to 
different aspects of school finance is evident. The equity issue of the 1970* s 
in funding local school districts has shifted to interest in funding formulas, 
budget increases and increasing teacher salaries. Of interest in the future 
will be financing issues of public versus private schools. Basic finance 
information concerning both sectors should continue to be considered a core data 
element in any elementary/secondary education data system. Particularly in the 
Private School Survey, attention should be given to the finance questions. To 
enhance their data base, NCES should also examine the finance data base 
maintained by ECS. 

Data Duplication/Overlap 

If the NCES data collections, as presented in Attachment B of the initial 
correspondence concerning the redesign, are examined by subject area rather than 
by data collection program some duplication of effort becomes evident. For 
example, if the topic of information on teachers is examined then it appears 
from a cursory review that several surveys collect information on teachers in 
addition to the data available from administrative records. We recommend a 
thorough review of the Survey of Teacher Demand and Shortage, Recent College 
Graduates Survey and the Public School Survey to examine duplication. All three 
ask questions of teachers concerning subject matter assignments. The Public 
School Survey and the Survey of Teacher Demand and Shortage both ask questions 
concerning teacher incentive plans. The Recent College Graduate Survey as well 
as he Public and Private School surveys obtain teacher salary and compensation 
data. 

In addition, there is duplication between NCES and other federal agency 
collections. For example, the Bureau of Labor Statistics, as part of the 
Occupational Employment Statistics (OES) program collects data every three years 
on current teacher employment. Another form of duplication is when information 
is collected in a survey but is already available in administrative records. 
Information on the number of new hires and the number of those teachers 
returning to their previous position from the survey of Teacher Demand and 
Shortage could potentially be obtained through the Employer's Quarterly Wage and 
Tax Report of State Employment Security Agencies (SESA). 
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We would recommend that because of the apparent duplication the Survey of 
Teacher Demand and Surplus be given a very low priority and reviewed to 
determine if it should be conducted at all. It seems that the data could be 
obtained primarily from the BLS OES program, state adminibtiali ve iecoius and if 
needed through the Public and Private School Survey. As well, the recent 
College Graduates Survey should be examined to determine what information is 
received that can't be derived from College/University and SESA administrative 
records and/or the Public and Private School Surveys. 

Although the area of teacher data seems to have the largest number of 
separate collections and therefore the greatest potentL . for duplication, other 
areas as well may also have inefficiencies. Beyond this one example cited above 
which needs further examination, we suggest that NCES do a comprehensive review 
of their data collections across subject areas to explore further efficiencies 
that could be realized through unduplicated data collection and more extensive 
use of administrative records. A single collection instrument that obtains 
relevant data for multiple purposes and users appears to be a far more efficient 
use of resources than multiple shorter surveys resulting in several sets of 
incompatable data. 

Unmet Data Needs 

A data system to remain relevant to users should be flexible in meeting 
data needs created by the changing nature of our society. Although several 
unmet data needs can be identified, no priorities have been assigned to these 
needs by state policy officals. For example, the High School and Beyond Survey 
traces the post-high school experience of graduates but not of those youth who 
are not graduating or who have dropped out. This will become an increasingly 
important topic to determine the experiences of the at-risk population and the 
impacts of the policy initiatives of the early 1980s. Again, administrative 
records may be a tool useful in gathering some of this data or it may be 
necessary to explore collection of these data through the High School and Beyond 
Survey. 

States need to identify education outcomes related to their own labor 
markets to fully use the data in their own policy development process, because 
of different industry structures, different rates of growth/decline and 
different labor market barriers and characteristics. For this reason, states do 
not find useful nations 1 longitudinal education data that only report national 
estimates, estimates for the 9 census regions and the 7 largest states. 
Although 6 or 7 states paid to have data collected for a state specific sample, 
generally there is no state level detail available. 

We acknowledge funding limitations and because of this are supportive of 
NCES efforts that would be more creative in developing arrangements to have 
states expand their samples. Also NCES should consider expanding the national 
sample to provide more state specific data. 



Another gap is information on career ladders within the teaching 
profession. Although there is a general lack of this type of information for 
most occupations, with the emphasis being placed on incentives for teachers, 
this information is valuable. Methods to obtain this data should be considered 
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by NCES working cooperatively with the National Occupational Information 
Coordinating Committee. 

Issues Thai: Crosscut Specific Data Frog iamb 

The NCES efforts to assure quality data is produced as recommended in the 
NCES "Research in Statistics" report are critically important. Governors and 
state policy representatives are seriously concerned about the accuracy of 
reported data. It is generally acknowledged that there are few validity studies 
made or audits done on NCES data collections. We therefore strongly support 
work to develop procedures and strategies for continually assessing the validity 
of all NCES data programs and encourage NCES to make this a top priority. There 
are other et forts that would also improve the quality of data; such as 
developing minimum definitions that would make data consistent across states. 

Standardization of Data Needed 

NCES data are most useful to state policy makers when comparisons can be 
made between states and between a state arid the national trend or average. For 
example, the indicators of education status and trends are excellent vehicle 
from which comparions can be made if the defintions used across states are 
consistent. Some states even indicated to us that it would be preferrable to 
have a less extensive data collection effort that contained more rigidly defined 
data elements that were released more quickly. 

States are very interested in assessing the amount and method of teacher 
compensation. In a survey of Governors' education policy staff conducted in 
November 1984 by the State Education Policy Consortium 43 percent of the states 
responding in the area of teacher quality indicated that the single issue which 
would be the most important over the next year is teacher compensation and is 
likely to remain so for the next decade. 

In this case, state specific information on the state education budget, 
current compensation level and structure, the state history of salary increases 
by local education agencies, comparable personnel costs within the state, the 
elements included in the compensation package, and teacher characteristics may 
be used. National information on the current national median teacher salary, 
whether it is rising/falling and by how much, the variance of each state from 
the median and where each state ranks compared to others gives the state a 
relative measure of their teacher salary program against a national indicator 
and neighboring states. Salary information should be collected in a consistent 
manner. 

Comparative measures that are not based on standardized definitions to 
guide data collection and that do not include a description of what's reported 
and what's not reported in the data may lead industry and education policy 
analysts to erroneous conclusions. For example, consider the detailed 
comparisons of data across states for the retirement system. In some states, 
local government pays the employer contribution and in others the state 
government pays the employer contribution. A state could have relatively higher 
or lower figures based on who pays for the retirement system and how the data 
are collected. Another example is enrollment data which, depending on the 
state, could represent average daily enrollment or a head count. 
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Numerous other examples exist of data collections where no standard 
definitions are used and data are not compatible across federal data 
collectors. An example is the long term debate on how to define a teacher. The 
BLS/Census and NCES definitions both differ. While we do acknowledge that there 
are different uses for different types of data, because of the need to integrate 
and use a variety of data sources in answering policy questions discrepencies 
such as this make it extremely difficult to accurately interpret the data being 
examined. Instead of using different definitions in various data collection 
programs we believe that it is the responsibility of the federal government to 
use common definitions for data collection. 

To overcome the definition inconsistencies such as these that render the 
data meaningless for the key purpose to which they are used at the state level 
will require a sustained long-term effort to determine what are the core 
elements and hew they should be collected. While education systems do vary 
widely across states; it appears that states would welcome common reporting on 
certain national data elements that would allow valid comparisons to be made. 

Fifteen years ago BLS had similar problems with tlie definition of 
unemployment. NCES may wish to look at the BLS federal/state cooperative 
prog rams 

as an example of how national definitions are used particularly with state 
administrative records. The BLS defines those data elements which are necessary 
for the national income accounts data. The BLS then contracts with the state to 
collect data using these standard definitions; using the dollars as a leverage 
tool. 

The BLS as the major statistical agency responsible for labor force 
statistics has defined the population (16 years and older) into mutually 
exclusive categories as shown in Figure 1. NCES as the major statistical agency 
responsible for education statistics, should consider defining the population 
(0-16 years old) in a similar fashion of mutually exclusive categories. This 
would help in the development of definitions. 

Standardizing data is not done without problems. BLS has been taken to 
court over the definitions but has won all the cases. Definitional problems 
should be carefully examined through more extensive collaborative efforts 
between the federal and state levels. A useful mechanism may be an interagency 
approach which brings together users and producers such as that of the National 
Occupational Information Coordinating Committee. 

Statistical Agency Coordination 

Under the authority of the Federal Paperwork Reduction Act we would 
recommend you pay particular attention to coordinating education financial data 
collected through the various programs with the Census Bureau which collects 
state and local fiscal data through the Census of Governments. The Census 
Bureau could potentially collect the data for NCES under an interagency 
agreement or contractual arrangement collection similar to the CPS October 
education supplement collection. 

Other areas where interagency coordination would be appropriate are with 
the Department of Labor *s Employment and Training Administration. Longitudinal 
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Figure 1 
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survey data from a variety of sources including the JTPA training programs would 
be more useful if coordinated among agencies and made more compatible. Under 
the JTPA, a longitudinal jcb training survey is conducted usirg a national 
sample of individuals representative of youth, all adults and welfare adults 
conducted on an annual basis. Data are collected on the individuals at 12 
months and 24 months after program completion. We would suggest here some 
sharing of information between NCES and the Employment and Training 
Administration. There may be questions of interest to NCES whi-zh could be added 
to the JTPA Survey and as well NCES could add questions of interest to ETA to 
their survey. The coordination of these sources of data would provide an 
expanded data base for more extensive use and would be a more efficient use of 
limited resources. 

Release Raw Data Quickly 

In many situations data are needed quickly for policy purposes. For 
example, if states want to change their salary level and/or structure in 
relation to the current national average trend, then data that are several years 
old are not useful. The early data systems being recommended by the NCES 
"Research in Statistics" report is one step to address the issue of timeliness. 
However, states will have limited uses for these data due to the small national 
sample sizes. State education policy makers would like quicker turn around time 
on the administrative data submitted to NCES directly by the states. An 
emphasis on technological initiatives that improve data collection, editing and 
processing procedures and data release capabilities will aid in shorter 
turn-around times. 

It has been suggested by several state data users that the administrative 
data collected from the states be made available in raw form to provide earlier 
access by state analysts. Use of NCES data may well increase if the turnaround 
time from the point when states report data to the point when NCES disseminates 
the collective data sent back to states is shortened. 

It was suggested by several states that NCES should concentrate les:> on 
detailed analysis of state-specific data or inter-state comparisons and more on 
national trends and a look to future issues. State experts who are familiar 
with the particular qualities of the area being analyzed and the differences 
between states should be primarily responsible for this analysis. 
Interpretation of national data should be conducted based on a set of standards 
which maintain the integrity of the data. 

Conclusion 

There is general support for NCES data products and publications in the 
states although this is not the primary source of data used for state education 
policy-making. The states as partners with the federal government are committed 
to the reporting of state/local data under reasonable requirements to various 
federal agencies such as NCES and national organizations. In some states this 
reporting activity may require up to 3 to 4 person-months per year. In 
exploring the return on the state 1 s investment in NCES data and the usefulness 
of the nationally collected state education data several key issues stand out 
from a state policy perspective. By far, the most important issues cross 
cutting a majority of states are that of: 
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(1) improving the quality of data; 

(2) providing comparability of data across states, across data programs, 
and across data collectors ■> 

(3) the increased use of state administrative records such as those 
available through the SESA; and 

(4) capitalizing on the statistical expertise in other federal government 
agencies and associations. 
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NATIONAL SCHOOL BOARDS ASSOCIATION 
1E80 Duke Street, Alexandria Vmnmia ^ 

June 27, 1985 t703r B 3B-NSBA 



Mr. Emerson Elliott 
Administrator 

National Center for Education Statistics 
1200 19th Street, NW 
Washington, DC 20208-1631 



Dear Mr. Elliott: 



We are pleased to comment upon the Elementary/Secondary Education Data 
Program proposed by the National Center for Educational Statistics 
(NCES). The redesign project 1s a welcome effort on your part, not 
only because 1t 1s commendable of NCES to consider the need to update 
Its efforts but also (and especially) because we appreciate the 
dedication and thoroughness with which NCES 1s seeking counsel and 
advice from throughout the community of educational statistics users. 
We hope that this NCES process will serve as an exemplar to other 
offices of the Department of Education. 

The most useful service to local school district policy makers, and to 
the state school boards associations that assist these local policy 
makers, is that federal education statistics capture significant 
trends 1n a timely way. The current effort 1s Imperfect, 1n this 
regard; 1t catches some but not others. For example, the movements 
toward magnet schools and toward desegregation are not readily traced 
1n NCES's data, while changes 1n the nature of the teaching profession 
are more readily found there. 

The next decade — because of the "excellence movement," the changing 
demographics of the U.S. population, and the emerging electronic 
technologies that many hope will Improve sodaty at large and schools 
1n particular — 1s likely to be an era of change m public education. 
Capturing trends, 1n a timely way, will require some reshaping of 
NCES's Inquiries. 

In light of this general conceptual background, we respectfully offer 
the following comments and recommendations: 

• We anticipate a continued policy Interest 1n Improving the 
effects of the schooling enterprise. We need to shift our 
statistical focus more than we have, to assist the public 
debate about school effectiveness. Certainly the public's 
Interest reflected 1n the "excellence movement" has been 
framed 1n terms of Increasing student learning -- not only 
academic learning measured by traditional tests but also 
other kinds of learning: The changing nature of employment 
has prompted attention to thinking skills, computer 
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capability, job-seeking and job-holding skills. Higher 
education Institutions have expressed concern about levels of 
writing and study skills. Some of the nation's social 111s 
have focused attention on a gamut of values held by students 
and graduates, ranging from entrepreneurshlp to patriotism 
and racial tolerance to sexual responsibility. The focus on 
effects of schools should reflect this broad range of 
learning. 

The current NCES program, however, generally focuses its 
attention on v,nat goes Into the schooling process. This 
historic Imbalance has an unintended consequence: the sheer 
weight of data reported about the number and characteristics 
of teachers, courses taught, attendance, enrollment 
categories, family characteristics, et al., leaves the 
Impression that education 1s more Interested 1n counting our 
resources than counting our effects. Moreover, the emphasis 
on statistics about the schooling process assures that 
process trends will more likely be captured than trends in 
the effects of education, so that the public debate gets 
frustrated. We hope that NCES through this redesign project 
will find new ways to aid the public's Interest 1n the 
effects of school 1ng. 

• New electronic technologies, and new systems for management 
and Instruction that exploit these new tools, will serve and 
reinforce this public emphasis on the effects of schooling, 
in two ways: 

- The arguments for and against uses of computers and 
other technology will be formulated on the basis that 
new tools do (or do not) In^rove the product of the 
schooling enterprise; and 

- Some of these new tools enhance the ability of school 
management to gather timely data about the success of 
the schooling enterprise. 

We need data about the uses and Impact of technology. There 
1s much talk and some considerable action 1n the uses of new 
technology but little useful data. Only market sales data is 
currently available routinely. The current ad hoc study by 
the Research Triangle funded by NCES on Computers for 
Instruction 1n Higher Education deserves a parallel study for 
elementary and secondary education. (Some of the questions 
1n that study, that focus on policies and academic 
requirements suggest some ways to address some of the issues 
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about the effects of schooling also.) It would be very 
useful to know net only what devices schools are buying, but 
also to what uses these devices are being put, how staff is 
being trained, how courses and budgets are being altered et 
al . 

Another aspect of technology's impact will be changes in the 
roles of school professionals. Current federal statistics 
seem to suggest that all teachers are alike; and that they 
work 1n classrooms. We expect to see further differentiation 
among school staff roles, fueled not only by pressures for 
career ladders and merit pay but also by the Introduction of 
technologies. Already, for example, the "computer teacher" 
that serves 1n a "laboratory" as a resource to many other 
teachers has a very different role from the conventional 
image of what teachers do. Other new professional roles and 
titles and circumstances are likely to emerge as schools use 
television, computers, electronic mall, on-Hne data bases 
and laser-disc-based data bases and other new technologies. 

Two other dimensions of the Introduction of new technologies 
can be anticipated: more diverse Instructional strategies 
and more diverse student roles, as the new mediated and 
individualized tools become commonplace. 

• The National Assessment of Education Progress (NAEP) 1s a 
vital federally-financed program because 1t provides a 
continuing measure of some effects of our schooling system. 
This, and perhaps other "snapshots" by NCES can give a 
picture of changing student attitudes (that 1s, attitudes 
that schools may have a role in shaping) to supplement data 
about academic learning. 

• Four topics about the schooling process are not well captured 
1n the current data program that should be improved 1n the 
next decade, as they are likely to be of policy concern: 

- courses available to students; 

- populations served by schools; 

- off-campus learning; and 

- uses of new technologies. 



We recommend special attention to these four topics. 

New adm1n1strat1vp procedures, and both old and new 
technologies, make 1t feasible to broaden the menu of courses 
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that can be taught even 1n the smallest schools. On the 
other hand, pressures to get "back to basics" and to shift 
academic priorities tend to reduce the menu of courses 
available In larger schools. What to teach, and how to make 
it available, are constant policy topics at local and sMte 
levels. Trend data would be helpful here. 

The federal statistics program seems to define public schools 
in K-iz terms. Yet schools everywhere are looking at 
pre-school care, after-school care, and many forms of adult 
education and services. Whom to serve, and how to render 
services to new populations, are policy Issues at local and 
state levels which NCES may be able to illuminate with trend 
data. 

The Issue of on-campus versus off-campus learning suggests 
several dimensions. Technology makes home-based learning 
more feasible and school-bu1ld1ng-based learning less 
necessary One trend 1s the growing Interest by some school 
districts In uses of broadcast, public and cable television. 
Another 1s the often expressed need for school/business 
collaboration; related 1s a potential for greater 
collabor?t1on between schools, libraries, museums and hlqher 
education Institutions. y 

In summary, schools are likely to change In several ways durlnq the 
next decade; the best NCES service would be to capture significant 
trends In a timely way. 

We see a most significant trend in the shift In the public debate from 
debating the process of education to debating Its effects. NCES data 
should also m?ke the shift to facilitate this trend. 

We hope that these comments and recommendations will be helpful to vou 
as you participate in the laudable effort to Improve NCES data for 
practical use In local school districts throughout America. 

Very truly yours, 




ERIC 



Thomas A. 3nannon 
Executive Director 

TAS/mk 

cc: William J. Bennett 

Secretary of Education 
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RESEARCH ANP DATA NEEDS FOR SMALL /RURAL SCHOOLS 



A Position Paper prepared for the 
National Center for Education Statistics 

In consideration of NCES's plans for the 
Redesign of their Elementary and Secondary 
Data Program 



by 

*Bruce Barker 
Box 4110 
Texas Tech University 
Lubboc^, Texas 79409 



June 17, 1985 



*Bruce Barker is an Assistant Professor of Education and Assistant 
Director of the National Canter for Smaller Schools at Texas lech 
University. He also serves on the Research Committee of the national 
Rural Education Association. This article has been prepared at the 
request of the Rural Education Association. 
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by 

Bruce 3arker 

Although discussion of research and development needs for 
small/rural schools is becoming mo~^ prevalent, the state of 
knowledge and information currently available on rural educatxuu 
remains incomplete and startlingly inadequate. The first National 
Seminar on Rural Education held in Washington, D. C. in May, 1979 
recogr/.zed the need to collect, analyze, and compile data pertinent 
to rural education (Flectcher, 1979/80). Tamblyn (1977) indicated 
that one of the major tasks in the 1980' s for rural education was the 
need to conduct basic research on small school problems, practices, 
and unique features. Horn (1981) declared that one of the 
responsibilities facing universities is to conduct research and 
collect daca on rural schools. And, Nachtigal (1979) specifically 
stated that descriptive data are needed on the operation of K-12 
r'ral school systems with enrollments of fewer than 300 students, 
300-999 students, and 1000-2500 students. 
The Problem of Definitions for Rural Education 

Lack of a precise definition may be one reason rural education 
has received little atcention in recent years. Rural education has 
been a difficult entity to define because the word "rural" has 
different meanings when viewed historically, statistically, or 
philosophically (Salmon, 1980). Furthermore, the concept of "rural 
educatior varies from state to state and region to region. Both 
Texas and Oregon, for example, define a small/rural district as 
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having fewer than 1000 students (Barker, 1985). Since 1970, the U.S. 
Census Bureau has carefully defined the rural papulation as 
consisting of all persons living in places of fewer than 2500 
inhabitants or in areas of extended cities with a population density 
of less than 1000 persons per square mile (U.S. Department of 
Commerce, 1970). The National Advisory Council on the Education for 
Disadvantaged Children uses the Census Bureau's definition to state 
that a rural district is, therefore, one having fewer that 2500 
students ( Special Reporc on Rural Education , 1979). Other 
definitions include those of the American Association of School 
Administrators which has established a K-12 enrollment of 2000 or 
less as a small district and the National Association of Secondary 
School Principals which considers an enrollment of 1000 or less to be 
a small high school (Jinks, 1984). 

Some of these definitions imply that rural America collectively 
consists of all our society's nonmetropolitan areas. Inference is 
also made that this portion of our society is basically homogeneous 
in nature and composition. In reality, rural America is a vast array 
of diverse nonmetropolitan areas which may be internally more 
homogeneous that most urban communities, but which differ widely from 
each other. For example, an island hamlet off the coast of Maine, an 
Alaskan native village near the Arctic Circle, a coal mining town in 
West Virginia, a ranching area in Wyoming, an impoverished community 
in the Mississippi Delta, a ski resort ection of Vermont, or a 
proz serous grain farming region in Iowa have little in common, except 
that they are all classified by the Census Bureau as rural areas of 
the United States (Sher, 1977). 
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The Lack of Attention Gi ven Rural Schools 

Federal statistics reveal that 59.5 million Americans live 
outside designated urban areas of the United States and that rural 
school students constitute the largest minority public school 
population in this country (Treadway, 1984; Sher, 1977). Based on 
the Census Bureau's definition of "rural/ 1 nearly two-thirds of the 
15,600 operating public school districts located in the United States 
are in rural areas and one student in every three attends an 
elementary or secondary school classified as rural (REA News, 1982). 
Ironically, however, the "lion's share" of attention, research and an 
over balance of federal and state financial support generally go to 
large schools in metropolitan areas. Not until late 1983, four years 
after the establishment of the U. S. Department of Education, did 
that federal department declare a Rural Education and Rural Family 
Education Policy for the 1980 's*' which stated, "Rural education shall 
receive an equitable share of the information, services, assistance, 
and funds available from and through the Department of Education and 
its programs" (ERIC CRESS, 1983/84). In 1983, tho National Center 
for Education Statistics also agreed, for the first time, to include 
small and rural schools of under 300 students as a separate category 
for data collection (REA News, 1983). Up until the time of these two 
actions, national policy makers and researchers had paid little 
attention to rural schools. 
Rural Education Research and Data Needs 

Among the expressed goals of the national Rural Education 
Association is to encourage "... the collection and dissemination 
of . . . statistical data and other appropriate information relating 
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to rural education" (REA, 1980). A National Rural Education Res arch 
Agenda endorsed by ths Rural Education Association Calls £ui research 
relevant to rural education in nine broad categories (Barker and 
Stephens, 1985) . 

1. Rural school effectiveness 

2. Staff development and professional support 

3. Curriculum and instruction 

4. Taxonomy of rural education 

5. Federal, State, and local policies impacting rural schools 
and communities 

6. Rural school finance 

7. School district governance and organization 

8. Assessment of rural school assumptions 

9. Role of the school in rural development 

These themes may not encompass all of the research needs for 
small/rural schools, but they do establish the major areas in which 
research is to be focused. Moreover, data collected in these areas 
will provide policy makers and rural school practitioners information 
to knowledgeably affect small/rural schools improvement. 

In the redesign of the elementary and secondary data program 
conducted by the National Center for Education Statistics, the Rural 
Education Association strong 1 * r encourages the inclusion of 
small/rural schools as a specific category in the collection and 
reporting of data. In light of the various definitions associated 
with rural education, it would seem that the collection and reporting 
of data on the basis of school district enrollment size would be the 
most utilitarian approach. Rural schools have always been, and will 
likely continue to be, characterized by smallness. According to Sher 
(1977), small public schools and small school districts hfive become 
increasingly rare in America's metropolitan centers. Urban schools 
and districts have generally always had larger student bodies than 
rural ones. With continued political and economic pressure to 
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centralize schools, the issues of the small public school will become 
almost exclusively rural, for rural areas will be the only places 
such public school exist in cignificanc numbers. 

In the collection and reporting of education data, the Rural 
Education Association recommends that NCES, whenever possible, break 
down the data based on school districts of fewer than 300 students, 
300-999 students, 1000-2500 students, and those in excess of 2500 
students. Such a classification would more accurately reflect rural, 
suburban, and urban similarities and differences. In addition, the 
availability of comparative data at the national level would provide 
policy analysts, public educators, and others interested in education 
with valuable information to assess American public education. 
Conclusion 

It is impossible to treat rural education as one single or 
common entity. Rural education encompasses everything from a 
one-room country schoolhouse in northern Vermont to a sparsely 
populated western school district responsible for education in a 
several hundred square mile region. It includes districts having 
solid financial resources and others with very limited funding 
sources. Some of America's fastest growing districts, as well as 
those experiencing the most rapid enrollment decline, are in rural 
areas. Because of this diversity, much of the effort put forth to 
improve rural schools can best be met at the local level, where area 
specific problems can be addressed and treated. 

It is not expected that the collection and reporting of national 
data on the basis of public school district enrollment size will 
solve the many challenges facing rural educators. Such information 
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will, however, enable local administrators to more knowledgeably 
assess the operation and management of their own school systems and 
will provide them with reference information on school systems of 
similar size. In our nation's quest for excellence in education, the 
data and information needs for small/rural schools must be included 
in any collection of statistics conducted by the National Center for 
Education Statistics. 
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SHEEO/NCES COMMUNICATION NETWORK 

Suite 31 0 • 1 860 Lincoln Street • Denver, Colorado 80295 • (303) 830-3687 



October 1, 1985 



Mr. Emerson Elliott 
Administrator 

National Center for Education 

Stat ist ics 
Brown Building, Room 606 
1200 19th Street, N.W, 
Washington, D.C. 20208-1404 

Dear Mr . Elliott: 



I am pieased to provide you with some ideas on the NCES project 
to redesign its elementary and secondary education data 
program. This is an important NCES initiative. We applaud the 
Center's effort to solicit advice from various persons and 
associations on how NCES might improve these data. The comments 
which follow were prepared by the State Higher Education 
Executive Officers/National Center for Education Statistics 
Communication Network Advisory Committee. I transmit them to you 
on behalf of the State Higher Education Executive Officers 
Associat ion. 



Most SHEEO agencies use national postsecondary education 
statistics available from NCES in a variety of ways. By 
comparison, their use of national elementary/secondary education 
statistics has been limited. Historically, this has been so 
because much of the elementary/secondary data used by the 
agencies has been obtained from their respective state 
departments of education. Many agencies have not taken advantage 
of relating national elementary/secondary data to national 
postsecondary data. Recently, however, the emphasis upon 
education reform at all levels within the states has generated 
the need to address interrelated elementary, secondary and 
postsecondary issues and problems. There are, therefore, soro* 
important general comments we feel need to be considered by NCES 
as the Center works toward redesigning and improving its national 
elementary/secondary data collection program. 

Many education policy issues in the states and at the national 
level bridge elementary, secondary and postsecondary education. 
NCES data collections from all sources and levels of education 
should be compatible so the totality of the educational 
enterprise and its continuity can be reflected in the data 
analysis. We need the capacity to assess changes in the 
educational process at transitional points along the education 
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continuum. The same definitions should be applied to NCES 
student and institution based survey data so students can be 
tracked from one level of education to the next. Analysts, as a 
result, should be able to reliably generalize sampled data to the 
population (i.e. the sampled data from NLS-88 can be gene alized 
for students in the IPEDS universe of institutions). 
Elementary/secondary data collections need to be compatible and 
linked to the NCES Integrated Postsecondary Education Data System 
(IPEDS) surveys to rrake possible the ability to interrelate the 
data. By interrelating the data bases, NCES will be able to show 
more clearly the multi- imensional character of education, and 
thereby demonstrate the need to address many educational problems 
at more than one area of the education spectrum. 

It is important that NCES improve upon both what data are being 
collected, and upon how the data are collected. Thus, another 
important element of th is redes ign effort is the use of current 
technologies for collecting, transmitting, and disseminating the 
data to be collected. The timeliness, quality and utility of the 
NCES data for researc\ers, administrators, state and national 
policy makers can be improved if the use of technology to collect 
and disseminate data is an integral part of the total redesign 
ini t iat ive. 

Beyond these general comments, there are some specific data 
elements, and related information that are needed by 
postsecondary analysts which should be a part of the NCES 
elementary/secondary education data collection program. The 
availability of such data will be of assistance to postsecondary 
education policy planning and development at the state and 
national levels. 

1. Enrol lments in public, private and specialized (state 

schools for the deaf and blind, etc.), high schools, and 
enrollments in school-sponsored, home-study programs. 
The enrol lment needed annual ly from the universe of 
school s include students in: 

a. Grades 7, 8, 9, 10, 11, 12 by sex, age and ethnicity; 

b. College preparatory, vocational and general 
curricular tracks for twelfth graders by sex and 
ethnici ty; 

c. Joint high school/collegiate level programs ; 

d. Third and fourth year English, mathematics, science 
and foreign language courses for grade levels 11 and 
12; 

e. Different kinds of remed ial courses and programs at 
the secondary r j postsecondary levels by sex and 
ethnici ty. 
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specialized high schools. The number of high school 
diploma recipients need to be available annually from the 
universe of schools by sex, age and ethnicity. 

3. A survey regarding elementary and high school personnel 
for the purpose of determining: 

a. Analysis of staff turnover in terms of "quit rates"; 

b. Percentage of high school teachers teaching out-of- 
f ield; 

c. Measures of teacher quality (i.e. experience, degrees 
held, test scores, self-reported grade point average 
in college, etc. ) 

4 . Other inf or mat ion to be provided through data analys i s 
(that perhaps can be derived from existing data) include: 

a. The participation rates of students from different 
types of secondary schools enrolling in different 
types of post secondary institutions; 

b. An analysis of the secondary school courses taken by 
former high school students enrolling in different 
types of postsecondary education institutions, and 
the relationship of such course work to previous 
academic achievement and test scores in elementary 
schools . 

c. Basic indicators of the progress being made in 
improving the condition of education at all levels. 

5. Analyses of student-based longitudinal studies that need 
to be continued include: 

a. High school drop-out rates from all kinds of schools, 
including private and specialized, by grade level, 
sex and ethnicity; 

b. Average achievement test scores for students by typ^ 
of high school (pub lie, private, special ized), 
location of school (rural, suburban, city), 

curr icular track (col lege preparat ion, vocat ional , 
general), by student sex and ethnicity; and 

c. Intent ions of high school seniors regarding work, 
military, or education upon completion of their 
secondary schooling with a six-month follow-up to 
determine the extent intentions are valid predictors 
of actual decisions made. 
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based data, provide essential information for particular 
purposes. Obviously, NCES should continue to collect such 
data. In Addition, student-based survey data are becoming 
increasingly important, iviany policy issues related to drop-outs, 
remediation, and student course work and achievement can only be 
addressed through the NCES-sponsor ed , student-based, longitudinal 
studies. The information gathered through these studies may, in 
the future, be the most important elementary/secondary education 
data series NCES sponsors. 

At its annual meeting in July 1985, SHEEO adopted several 
reconmendat ions it received from its SHEEO/NCES Network 
Representatives following their national meeting in June (see 
attachment). Several recommendations relate to improving the 
relevance, technical quality and utility of NCES data programs to 
better serve education policy makers at all levels. 
Implementation of these recommendations will improve NCES 1 
elementary/secondary and postsecondary education data 
collections. Reconmendat i ons 1, 5, 6, 14, 15 and 16 should be 
considered when redesigning NCES 1 elementary and secondary 
education data collections. 

On behalf of the SHEEO Association, I thank you for this 
opportunity to provide our thoughts on the e lementary /secondary 
redesign project. Please contact me for further elaboration on 
these comments if needed. 

Sincerely, 



John R. Wittstruck, Director 
SHEEO/NCES Communication Network 

JRW:as 

cc: Kenneth Ashworth, SHEEO President 

SHEEO/NCES Network Advisory Comnittee 
Leslie J. Silverman, NCES 
Richard C. Taeuber, NCES 
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U.S. EQUAL EMPLOYMENT OPPORTUNITY COMMISSION 
Washington, DC. 20507 



Mr. Leslie J. Silverman 
Deputy Assistant Administrator 
Division of Statistical Services 
National Center for Education Statistics 
Department of Education 
Brown Building M13A 
1200 19th Street, N.W. 
Washington, D.C. 

Dear Mr. Silverman: 

The U.S. Equal Employment Opportunity Commission is pleased to participate 
in the National Center for Education Statistics* effort to re-evaluate 
and redesign its data collection system. We applaud your long-term plans 
to improve your responsiveness to the needs of various users by providing 
"cross-sectional and longitudinal data relevant to policy issues and ad- 
ministrative needs, as well as to measurement of our Nati Vs education 
systems. 11 

Our input to the project consists of two parts. First, with regard \o the 
kind of data that NCES might consider collecting, we are providing the 
following general and specific recommendations whose thrust is toward the 
increased u~e case studies and ethnographic research: 

1. Collecting data at elementary and secondary school 
levels indicating actual enrollees by race and ethnic 
categories; 

2. Developing qualitative indicators of primary and elementary 
school preparation showing self-perception, motivation to 
learn, and orientation toward school and the world out- 
side of family and neighborhood; 

3. Providing interpretive analyses of Digest of Education 
Statistics tables and data sets; 

4. Collecting data at both elementary and secondary school 
levels on drop-out rates by race and ethnic categories. 
Also providing case studies of successful retention 
programs, incidences of high drop-out rates, and number 
of drop-outs who return and complete their studies; 

5. Ethnographic studies, particularly in urban areas, on how 
minority 3tudents "move through" the system, with emphasis 
on barriers and "tracking;" 
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6. Case studies of the correlation between levels of edu- 
cation and employment by race and ethnic group, in 
terms of income lev o1 s, unemployment and underemploy- 
ment ; 

7. Collecting data on Suudents "tracked" during elementary 
and secondary school as enrollees in general education 
curriculum, vocational education curriculum, or col- 
lege preparatory curriculum; 

8. Case studies of "more effective" and "less effective" 
school systems and programs ; 

9. Case studies comparing public and private schools — 
in terms of enrollments by race and ethnic group, 
quality indicators, objectives, value systems, 
teaching techniques, administration, and school 
organization . 



Second, in response to your invitation to submit papers addressing ^elect 
education topics, we are including with this letter two separate papers 
that discuss the relationship between equal employment opportunity and 
equal educational opportunity. More specifically, the papers address 
the critical issues of family, education and employment, with particular 
emphasis on minority communities. These essays, as well as a third paper 
that will be sent to you next week, serve as the foundations on which the 
above recommendations rest. 

We ask that you review these papers as "drafts" and not consider these 
as reflective of official Commission policy. Our intention is to pro- 
vide research papers for the purpose of stimulating public debate on the 
nexus between employment and education. 

^ any further information regarding any aspect of our package, please 
c^ ..act the respective authors or Mark. Wong at 634-6750. 




Enclosures 
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NATIONAL SCIENCE FOUNDATION 

WASHINGTON DC 20550 



Directorate for Science 
and Engineering Education 



June 19, 1985 



Mr. Leslie J. Silverman 
Deputy Assista.it Administrator 
division of Statistical Services 
National Center for Education Statistics 
1200 19th Street, N.H. 
Washington, D.C. 20208-1401 

Dear Les: 

I regret that I was not able to attend the May 28th meeting to discuss the 
redesign of NCES' elementary and secondary education data program. In my 
absence, Iris Rotberg represented NSF at that meeting. 

In response *.o your request for papers, NSF is sponsoring two projects at 
the National Academy of Sciences/National Research Council (NRC) that are 
of direct relevance to your review. One of these studies represents a 
year-long effort by the Committee on Indicators of Precollege Science and 
Mathematics Education, which was charged with proposing a framework for an 
efficient set of indicators, filling n the framework to the extent 
possible with existing data, and suggesting data and data analyses that 
will be needed in the future for a continuing portrayal of the condition of 
precollege science and mathematics education NSF is presently supporting 
a successor committee under the chairmanship of John G. Tr »\all of the 
State University at Stony Brook, which is addressing the important goal of 
developing imaginative new indicators. Jay N el or* NCES has been attending 
the meetings of the successor committee and ha a copy of the initial 
report. 

The second project concerns the supply, demand, and qualifications of 
teachers of science and mathematics. Under NSF sponsorship, the Committee 
on National Statistics held a conference on August 9-10, 1984 to identify 
problems with the available data, or gaps in the data, to discuss problems 
and possible implements in the models now used for estimating and 
projecting supply and demand, and to suggest activities for a follow-on 
study. Frank Corrigan of NCES has a copy of the conference report and, as 
you may know, NCES is presently in the process of transferring funds to NSF 
to support part of a new effort that will be based on recommendations from 
Lhe August 9-10 conference. 

I am enclosing copies of the introductory sections of bcth of these reports 
that you mav wish to use in addition to this letter, as official 
submissions for public comment. Of course, the NRC activities as they 
progress will probably have important implications for the NCES redesign 
effor f s. 
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In terms of specific comments about NSF's present needs from NCES 1 
elementary and secondary data program, I have the following comments: 

1. NCES data on teacher supply and demand and course offerings and 
enrollment are not broken down in sufficient detail to be useful to NSF for 
planning purposes. The notable exception is the 1982 transcript data from 
High School and Beyond, which produced enrollment data on detailed 

of science courses (such as physics, chemistry, and earth sciences) and 
mathematics courses (such as algebra I, algebra II, geometry, trigonometry, 
and calculus) . 

2. Time series data on course enrollment in science and mathematics 
disciplines are largely lacking. NCES should have more consistency in the 
design of surveys, data collection, and analysis. Also, more and better 
data are needed on the amount of time students spend on homewc <. 

3. The most significant determinant of teacher demand projection, are 
turnover rates vhich appear to be age specific). Yet NCES data on teacher 
turnover rates are several years out of date and even these earlier data 
are not age specific. Again, NCES projections of teacher demand are not 
broken down by science and mathematics disciplines. 

4. Supply projections are largely dependent on new teacher graduates. 
NCES uses the questionable practice of projecting new teacher graduates 
based on estimates of percentages of total bachelors degrees granted and 
these estimates are aggregated so that data on science and mathematics 
disciplines are not available. 

5. Almost no data are available n the reserve pool of teachers and 
th3 number who return to teach. 



6. The NCES practice of counting ceacher vacancies leaves a lot to be 
desired in trying to determine the extent of shortages of qualified 
teachers of science and mathematics. For example, there is evidence that 
many science and mathematics teachers are teaching out of-field because of 
shortages. Also, it is not clear whether a teacher certified in both 
mathematics and chemistry would be count", as a mathematics or a chemistry 
teacher, or both. 

7. Adequate information is lacking on the qualifications of teachers 
who are responsible for teaching science and mathematics in high school, 
middle school, and elementary school, in many instances, certification is 
not a good proxy for teacher qualifications because of disparate 
certification practices of states. We also need * n have more data on these 
state certification practices. 

8. In terms of curriculum content, periodic surveys should be 
conducted of use of various science and mathematics textbooks at each grade 
level. Surveys of textbook use should be followed by content analysis of 
the most frequently used textbooks. 

1 f) 2 
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I appreciate the high degree of cooperation that you and your 
collegues at NCES have had with NSF in the recent past and I 'ook forward 
to working with you on these issues in the future. 

Sincerely yours , 

Richard Berry 
Program Director 
Studies and Analyses 

Enclosures 



cc: W. Gillespie, SEE 
I. Rotberg, SEE/OSPA 



ERJC 
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THE OFFICE OF THE ASSISTANT SECRETARY OF DEFENSE 

WASHINGTON. D C 20301-4000 



FORCE MANAGEMENT 



1AUG 1985 



AND PERSONNEL 

(Military Personnel & Force Management) 



Mr* Leslie J. Silverman 

National Center for Education Statistics 

1200 19th Street, N.W. 

Washington, D.C. 20208-1401 

Dear Mr* Silverman* 

As we discussed at our July 19, 1985 meeting, the Department of 
Defense would be pleased to participate in the Elementary /Secondary Education 
Data Program Redesign Project and we have specific data we would like you to 
consider collecting on our behalf. 

Our secondary school data collection request, outlined on the attachment, 
8 terns from our need for current, consistent and reliable data in support of 
military recruiting. Specifically,, a key portion of the Recruit Market Network 
IRMN), a major DoD data base available to recruiters through a nationwide 
teleprocessing network, includes information about the high school population. 
Because the high school data for inclusion in the RMN is derived largely from 
recruiters, private firms, and secondary sources, w?. would be delighted to 
replace these sources with your data. 

If you have questions about the attachment or plan to host future meetings 
on this topic, we would appreciate your contacting Zahava D. Doering, Chief, 
Survey and Market Analysis Division, Defense K^npower Data Center, at 696-5826. 
Dr. Doering, or members of her staff, will attend relevant meetings and can 
provide additional information. 



Sincerely, 




Anita R. Lancaster 
Assistant n irector 
Accession Policy 



Attachment 
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HIGH SCHOOL INFORMATION AND DATA TO SUPPORT MILITARY RECRUITING: 

REQUIREMENTS 

Background 

o The Recruit Market Network (RMN) is a common data base, available to 
users (on-line) through a teleprocessing network, established to support 
recruiting efforts in the Military Services. 

o A key portion of the RMN is devoted to information about the nation's 
high school population, organized in ways to aid decision makers in allo- 
cating their resources. 

o The cataloguing, tabulating and associated problems of determining 
school locations, current enrollment characteristics, and public and private 
school inventories has remained an on-going problems for recruiters, mana- 
gers, and DoD officials. 

o Data currently available from the National Center for Education 
Statistics (NCES) does not provide the required detailed data. 

Objective 

o Provide the recruiting community, through the RMN, with a reliable, 
up-to-date, efficient, data base containing information on the nation's 
high schools. 

o Eliminate the need to utilize recruiters, private firms, and secon- 
dary data sources for this information. 

Requirements (each school) 

o The RMN requires a data file with a record for every high school 
which provides the following information: 

- Type of School 

— Public 

— Catholic 

— Private (Non-Catholic) 

— Vocational 

- School Location 

— City 
County 
State 

— Zip-Code 

— School District 
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- Senior Class enrollment counts, by sex and race/ethnic 

- Junior Class enrollment counts, by sex and race/ethnic 

- Previous Year's graduates counts, by sex and race/ethnic 

- Percentage attending college (for previous years 1 graduates) 
o These data elements are needed on an annual basis 
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